Here's my hypothesis.
Most things that go in your inventory are of class Object. When you harvest a crop that can be coloured, the resulting crop is a ColoredObject (subclass of Object).
When you buy a regular item from the travelling cart, you receive an Object, regardless of whether the item is normally coloured or not.
When stacking items, it appears the intent is to not be able to stack an Object with a ColoredObject. However due to how it is implemented, this can happen. [Code: if ((other is Object && this is Object) || (other is ColoredObject && this is ColoredObject)) ]
Without testing it, it appears that this stacking can go either way, as long as the harvested items are of basic quality.