You can't 'reject' feeding them.
Only leave them alone.
Feeding has a window (the length of this dependis on the plant and pot). For saplings it is ~2 RL days (iirc, it has been a while since i did any light or dark ores).
Over the feeding window the plants will be weak (and feeding them strengthens them) so you need to check them at least once per RL day, if you don't they will die. (normally they can got 2 days between checks)
IMO light and dark ores are a pain in the ***. You have to give you plants more checks as you aren't feeding them and they take longer to grow (as you aren't feeding them).
Although feeding is a bit of a pain (trying to feed on the right day), I simply plant on Friday afternoon (Darksday) and as each feeding point is a little over a week apart my feeding days will fall roughly on Saturday morning (first feed), saturday afternoon (second feed) and sunday (harvest).
Because they are always at the weekend it is relavely easy to wait for the right ingame day (not limited by work etc).
This is worth a read