Newly Sprouted Oak

[Lazer32]

I have planted a few newly sprouted oaks, and they have been sitting there for weeks still looking like just a sprout. I figured it was because I planted them too close to other objects like walls and trees, so I decided it was time to remove them and plant elsewhere. However, for some reason I can't "chop" them away. I get the option to chop the little bastards but when I click on it nothing happens!

It is definitely not a pressing bug though, I mainly want them removed for aesthetic reasons.