We aren't doing too well for details, but I'll make some guesses. When you look downward at the retaining nut in its recess, is there a small amount of room around it? Usually you need to use a socket spanner to unscrew the nut: there should be just enough room around the nut to accommodate the correct sized socket. You will have to restrain the engine from turning while you loosen the nut. You do that by holding the blade plate (underneath the mower - the part that the blades screw onto). Never, ever put your hand under the mower without disconnecting the spark plug lead and making sure it cannot fall back onto the spark plug - jam it between cooling fins, or whatever.
You may not be able to hold the blade plate strongly enough with your hand. If necessary attach a G-clamp, pair of vicegrips, or whatever to the blade plate so that it catches on the mower base and stops the engine from rotating.