Roman legionaries stationed in the modern Middle East had a reputation for going without their armor due to the heat, but the Romans took the issue seriously enough that men could, theoretically, be executed for not being fully equipped while on duty. In addition, there are accounts of Roman commanders in hostile territory giving general orders to stay in armor essentially at all times - though none to my knowledge explicitly address labor details in armor. There are depictions (though such stylized propaganda must be taken as not necessarily literal) on the Column of Trajan of armored soldiers in extremely hostile Dacia doing labor details in full kit. (see here, about the middle of the photo)
I somehow doubt the romans usually dug ditches in full armor
Usually, almost certainly not. However,