FPS stands for frames per second, and is a measurement of how fast a computer can run a certain game. In simple games the update and rendering speed are tied together, as in update, render, update, render, and so on. Let's say we limit our game speed to 50 updates per second. This means that 1. the game cannot be rendered at a higher frame-rate than the game is updated, because an update is followed by rendering, meaning that the game cannot render at any higher than 50 FPS, and 2. that if the rendering of the game (which is usually heavier than updating) starts taking too long time the whole game will run slower. It will not just stutter, it will actually run slower, which is often undesired. It's better to not render each frame in this case and instead focus on updating so the game actually runs at normal speed.

For a better explanation see the excellent game loop tutorial here: http://www.java-gaming.org/topics/game-loops/24220/view.html.

To summarize: You often want your game to render at a rate that is independent to the updating rate.