According to wiki and forum research, a mine hole also acts as a mine support with radii equivalent to the wooden one and centered at the ladder.
So... how did these cave ins happened?
Note: They are caused by me removing another support, but (according to wiki) that still shouldn't cause cave ins since those tiles are still supported.
Also note: I also turned on Display mine support radii and it might be hard to tell in the pic but it's not showing