Attackers: Sat around, watched the castle/city, sat around, burned down neighboring undefended castles/cities, looted, prepared weapons, sat around, maintained equipment, drilled, sat around, pillaged, raped, sat around.
That sort of thing. Plus a lot of sitting around watching for defenders trying to escape/attack the sieging army.
Defenders: Sat around, watched the enemy, sat around, attempted to mess up whatever preparations they had to storm the walls, sat around, starved, died of diseases.. basically the same thing as the attackers, but it sucked more.
Spoiler
Note: These is my relatively uninformed conclusions from what an army could logically do whilst relatively chained to a single location. Do not take what I say as fact, as I could be entirely wrong, and they simply spent all their time sitting around.