Executive Summary
If your GPA is 3.99 it is a corner case, and you probably shouldn't round. If your GPA is 3.29, just list it as 3.3 in most cases.
Common Sense Rounding
My personal belief is that rounding is the best way to present information if done in a common sense way. Unfortunately, as most things with numbers, common sense is anything but common.
Value of a Decimal
Let's say my previous job was selling Boeing Dreamliners. A Dreamliner sells for about $206.8 million.
After haggling with a customer about options and pricing, we come to a verbal agreement to sell for $207,126,436.82
When presenting to management for approval of the sale price, most of the time top management won't care about the pennies. If you say $207.1 million, you have left out $26,436.82 of the price. Realistically, it could be $49,999.99 different. On a $207.1 million purchase, $50k is 2.414292612264606e-4, or in easier to read terms, 0.02% of the overall price.
The more numbers you throw out there, the harder it is to process, and the more time spent looking at number the more important the details of that number become. This is closely related to Parkinson's Law of Triviality.
When considering rounding, consider how much "value" you lose (0.02% in the case of the Dreamliner) as a consideration for whether to round, and how much to round.
Round up with Caution
Humans are dumb with numbers. That's why prices so often end in .99. That's why we guess high when given a high anchor (and vice versa). One of the consequences of this stupidity is that we are a lot more sensitive than we should be when we move up a digit.
So always round up with caution when it will make the left-most digit bigger. If you run a coffee shop that doesn't deal with pennies (rounding up), for instance, then you're probably better off not pricing something at $0.96 or above, because a customer expecting a cup of coffee for under a dollar will likely be unimpressed when the total comes to $1.00. It's just a psychological sucker punch.
So even though rounding is generally good, even if the overall impact is minimal (2% on a cup of $0.98 coffee!), the psychological impact can feel bigger. When rounding, make sure you aren't going to punch someone in the gut with it.
Guiding Language
Let's go back to the Dreamliner example. I made 5 sales, average price of $206.8 million each. That's $1,034,000,000.00 in total sales. How can I best write this in to my resume?
First of all, I don't want to write out $1,034,000,000.00 and have the reader need to count commas to figure out how big of a number that is. I definitely want to round that to $1 billion (losing $34 million, or 3.4% of my total). So I could write:
Generated $1 billion in revenues
But there is an issue here. The reader may think I'm being tricky. What if I really only sold $950 million, and am rounding up? To make sure that the reader doesn't think I'm trying to mislead them, I should make sure to help orient them to make the correct assumption:
Generated over $1 billion in revenues
By doing this, even though I rounded off 3.4% of my sales revenues, I end up gaining somewhere between 0 and 4.9% depending on how they think I rounded (heck, they may even be generous enough to think I generated 1.09 billion and didn't want to add 2 decimal places).
Whenever presenting numbers that are rounded, do your best to do language that guides the reader in the proper direction to eliminate confusion and thwart number-sticklers who love pointing out any small little discrepancy as it gives them joy in the same vein as grammar fascists.
Know Your Audience
If you are applying for a job as a rocket scientist, you may want to value accuracy over ease of reading (maybe). After all, while ±0.02% may not seem like much, in many engineering fields that's utterly unacceptable. Same is probably true in accounting (I don't think Boeing would want an intrepid accountant taking that additional $26.4k off each airplane sale and putting it in his own account).
But when there isn't a specific detail-oriented audience, common sense rounding should serve you well.
Trust Your Gut
After you're done rounding, think about how you would feel explaining the justification behind your rounding. If you don't feel comfortable, then don't do it. For instance, if a job is asking for minimum 10 years experience designing widgets, and you have been working at a company for 9 years, 7 months, but spent your first 6 months working on cogs, you may take pause at writing "10 years experience designing widgets" since your gut tells you it's really close to 9.
So what the heck does this have to do with my GPA?!
So let's apply common-sense rounding to your GPA.
Rounding 3.99 to 4.0 is rounding to a single decimal place, giving a maximum difference of almost 0.05. On a 4 point scale, that is ±1.25%. No problem on that front.
But when you round 3.99 to 4.0, you are increasing the leftmost digit, which is a psychological sucker punch ("you implied you're perfect, but you're not! How dare you violate our trust?!").
If you are going to round then, you would want to round down to 3.9, but guide them in the right direction by saying >3.9.
In the end you are left with your original:
GPA: 3.99
or the Common Sense Rounding approach:
GPA: >3.9
Which is better? Well, both are 3 digits, and the 3.99 makes for an interesting conversation starter, as said by a few folks here, so I'd just leave the 3.99. But realize that this is a corner case, and in most other cases I'd advocate rounding up to a single significant digit (because honestly, what's the weight of a candidate with a 3.2 vs. 3.3 GPA? Will it really factor in to anything?).