I'm surprised no one's chimed in with a big academic post about this.
In general, you want classes to be responsible for their own inner workings, and only allow public access in limited ways. As long as they do their job, other classes shouldn't really care about how it's done. There's a couple of reasons for that.
As an example, think of an OrderedList class that you might write. There's a few public methods that you would naturally want it to have - add(item), remove(item), getFirst(), getLast() etc. Outside classes don't need to know about how the list is stored and maintained.
You might decide to store all the items that are added in an array, and you would want that array to be private. If it was public, other classes could get in and manipulate it in ways that could interfere with how you are maintaining the order, or break certain references that you might be keeping internally.
And then later on down the track you might decide that an array is a poor way of handling it and decide to use a linked list instead. All the public methods (add, remove etc) can stay the same. Changing how the data is stored internally won't break any other code because the public access points are still the same.
If you'd made the array public to begin with, the temptation would be there to access it directly from other classes, which means changing to a linked list would break that code.
It's not super important if you're the only person working on a project, but it's good practice and it can save you some headaches from time to time.