On the mail server front, I dumped Exchange in favor of hMailServer, an open source e-mail server which handles the load of mail I get with less setup than Exchange required, less server resources, and generally less hassle. There are a lot of advanced Exchange features that hMailServer doesn't have, but I never used any of those features, so this is just fine with me.
For a mail client, I dumped Outlook 2003 (after looking at the Office 2007 beta, Outlook 2007 was never a serious contender for me) and went with Mozilla Thunderbird. This was a bit more painful; Thunderbird is rather lightweight in many areas compared to Outlook. On the other hand, the "lightweight" also extends to resource usage, and it's nice to have a mail client that starts quickly and doesn't drag the whole computer down. I imported my last six months' worth of mail to Thunderbird; the rest of the last five years is still accessible to me through a Copernic index if I need it, though giving up that archive is going to be a bit painful if I don't eventually import it all.
Dropping Outlook also meant getting rid of JetBrains Omea Pro, which I was using for RSS and general organization. After looking at a few alternatives I've settled on NewzCrawler for an RSS reader. Because of Larkware, it's important for me to keep a client-side archive of RSS posts, so all of the online readers are out for me, no matter how sexy they look. I'd ultimately like to replace NewzCrawler with something cross-platform but haven't found anything decent yet.
There are still a few pain points. Right now I have my task list in Evolution on the Ubuntu box, which is OK but not great; I'm so far pretty unimpressed with Mozilla Lightning for calendaring and tasks. So I'm not sure what I'm going to do there, but I can live with having things spread across two boxes for a bit. And I don't have things synchronized to my PocketPC either, though to tell the truth I wasn't much using it anyhow. Also missing is SharePoint calendar integration, important because for the moment we run our household calendar on a SharePoint server, though in the long run that won't matter because I'll probably replace that server with something else too.
All in all, moving to a different e-mail setup proved to be far less hassle than I expected it would be - which gives me hope for migrating some other things to non-Windows platforms as well.