A distributed computing project needs a problem that can be chopped up into little bits and, well, distributed. The little bit each of us gets has become known as a "work unit". It is just the right size - reasonable to download, but containing enough "stuff" so that our program can spend a lot of time doing worthwhile calculations. We call the actual calculations "crunching", a venerable bit of computer slang.
In GIMPS, a work unit is a complicated discussion. There is more than one "kind" of work unit and the statistics for these are kept separately. You can almost think of GIMPS as two distributed computing projects in one.