Be sure to use a Scholar’s Lens to see who built your walls. One of the oldest tricks in the game is putting your walls in someone else’s base and then using them to break in when the person isn’t online. That is almost certainly what just happened to you.