Migrating from Microsoft Exchange to Google Apps (Part I)

Recently I embarked on upgrading our existing Microsoft Exchange server to Exchange 2010. I was very impressed with many of the new features on 2010, most notably the new Outlook Web App and the high availability features.

The situation here at work is we are a small private college. We have around 120 FTE staff and faculty along with 500+ students. And throw in some emeritus faculty and we have a good sized Exchange installation. As I progressed through the initial design, installation and testing of Exchange 2010 things were going reasonably well. The initial testing revealed that our current version of NOD32 on the clients would delete all HTML e-mail. That was a shock and took some time to determine why e-mail was simply disappearing. Next we had to purchase some additional licensing for Windows Server 2008 R2 and Exchange 2010 to support the design that I had laid out.

Here is what our design looks like: We have a single CAS server running Black Berry Enterprise Server, OWA and hub transport. On the back end, I have two mailbox servers hosting the mailboxes. This will generally improve uptime for our clients and offer superior redundancy with the mailbox databases which is very important.

Things were going well, until I decided to check on CAL's for our students and emeritus faculty. Microsoft said that we must "refresh" our student CAL's for the move to 2010. Not good since budgets are tight.

What to do?

I initially thought that we could host a second solution on site using an open source groupware system. There are many out there that would fit the bill so to speak. Yet, this would increase complexity and we would have additional infrastructure to manage. I wasn't pleased with this solution. I checked into Google Apps for Education. Google offers educational institutions all of the Google apps for free. This seemed like a good fit, but would it meet all of my requirements?

Overall my requirement list is short:

1. Support single sign-on.
2. The ability to script account creation and deletion.
3. Simple back-end management.
4. Offer a good experience for our students.

Single sign-on, or SSO is critical to a seamless user experience, which is why it is listed first. Also, offering a good user experience is critical as well, but this is a known quantity since many people in the office use GMail, including yours truly.

The other two items on the list, scripting and management are more internal IT tools offering nothing to the customer. Yet, it makes our life simple in the office. Currently all account creation and deletion is handled with scripts that contact our ERP system to enumerate the list of students. I can easily create hundreds of accounts by executing three scripts. Not bad, but I'm working on reducing that down to one. The Google API is key to making this happen.

In the next installment, I'll discuss how I implemented SSO using Google Apps.

Removing malware from a computer

One service that seems to crop up is how to remove spyware, malware, adware and other nefarious forms of malware from a computer. So, I decided to write a general how-to guide on how I go about removing these nasty little programs from a Windows computer. First a little caveat, most of these tools have problems on x64 versions of Windows. I'm sure that the tools will progress but for now I'll concentrate on i386, 32bit Windows.

As with any trade one needs to have many tools in their tool kit. To remove these nasty programs takes a set of tools and not one. Why? It would seem reasonable to an outsider that a single program and/or suite would do the trick. Remember, it's hard to drill holes with a table saw and it's hard to make strait cuts with a drill press. Newer malware will snap themselves in as hidden device drivers, or other services marked as safe mode to fool AnitVirus and other spyware removal tools.

Here are the tools that I use:

• ComboFix (get this from http://www.bleepingcomputer.com/combofix/how-to-use-combofix and NOT www.combofix.org!)
• Microsoft's Malicious Software Removal Tool
• Windows Update
• Spybot Search & Destroy (http://www.safer-networking.org)
• AdAware (http://www.lavasoft.com)
• MyDefrag (http://www.mydefrag.com)
  

Start out by uninstalling any AntiVirus software that may already be on the machine. Why? Well, it didn't help anyway. More on that later.

Once all of the old AntiVirus software is removed, run Combofix on the computer. This tool alone will remove an incredible amount of junk. This creates a great starting point for getting the machine back in working order.

An old trick is for hackers to hijack your hosts file. This file can override address lookup and direct your browser to a different site without your knowledge. This file is located in the Windows directory, which is usually C:\Windows. You may use Notepad to edit C:\Windows\System32\drivers\etc\hosts. The only line that is needed in this file is this:

127.0.0.1 localhost

If you have more entries then this your host file has most likely been hijacked. You can edit this file and reduce it just down to the line above.

Next download and run Spybot Search and Destroy. DO NOT install TeaTimer. This tool will need to be run on all profiles residing on the machine. Next immunize the machine using Spybot. This will help keep away the bad stuff. From the advanced menu you can also view what is starting on the machine and uncheck items that are not needed.

Spybot has a good host file hijacking system where they can put in entries that will blunt any malware on the machine. I do this as a good course to prevent any malware from calling home.

Also from within Spybot you may run the system check. This will help to clean up the registry.

Reboot the machine and run Windows Update. This may take several passes and reboots to get all of the required patches installed. This alone will go along way to keeping the hackers out.

Once the machine has all of the Microsoft updates, run the Malicious Software Removal tool by running MRT. A quick scan is usually sufficient.

Then, download and run AdAware. This may or may not turn up more malware. Simply follow the instructions.

At this point the machine is clean, but may still be slow. Download and install MyDefrag. Once installed run the Slow Optimize. This will take a long time on some very fragmented computers. This is a good thing to start and then head to bed.

The final step is to install some legitimate Anti Virus software. I recommend Nod32 from eSet systems (http://www.nod32.com). I do not recommend Norton or McAfee, however, if you have paid for them and your subscription is current put it back on the machine and run it until the contract runs out.

If you don't like to pay for protection, AVG from GriSoft is an excellent free (as in beer or lunch) AntiVirus program. You can download the current one from (http://free.grisoft.com). They have a paid version too if you feel better about paying for protection.

Scan the computer for viruses, just to be sure.

Make sure that Automatic Updates is setup and running.

Keep in mind that these procedures will take a considerable amount of time. But in the end the machine will be clean and running like new again. In addition it will be hardened against possible future attacks.

I hope this helps some folks out. Stay safe out there!

Job hunting in a down economy

For the past few months I've been actively job shopping. Due in part to a demotion and a pay cut. But times are tough, but not as bad here in the mid-west, unless you work in aircraft. I've had a couple of job offers that I've frankly had to turn down. They were good offers with good companies.

Some wives are great at holding up mirrors and keeping their husbands somewhat in check. Mine is one of those. I'm normally a rational and pragmatic guy. I have to be, I work with computers. Computers don't care if I hope that a piece of code will work or not, either it does or it doesn't. Systems are the same way. Life in IT is generally a two-state system. Either the computer/system/program works as it should, or it doesn't. But I'm an emotional creature too with a bit of an ego.

Yes, being offered a position is a bit of an ego booster, and being handed the "thanks, but no thanks" letters is a bit of a let down. I just got another one of those too. So why turn down a position?

The answer is not as simple as it would appear. It is not always about the salary or the position. Benefits and stress play a huge portion of the equation too. Some benefits are tangible, 401(k) contributions, paid time off, vacation time, etc. Others not so much, the boss, coworkers, challenges within the job and overall stress. And, I'm not getting any younger. Not that I'm slowing down, but my direction on life is in a much different place than it was 10-15 years ago.

Salary and title are not the single most important items on the list. What is important? Stress, relationships with coworkers, bosses, and the mission of where one works. Taking a step back the latter seems to be more important than the former.

Overall where I work is still a good place to be. The benefits beyond the salary seem to out factor just the salary alone. Some would say that I'm lucky to have a job. Luck is the law of probability taken personally, I don't believe in luck. I know that I'm talented and frankly I just need to calm down a little and all will be okay. Irrational fear is not a good thing. Always look at things objectively keep focused and things will be okay in the end.

The laws of unintended conciquence

I wrote earlier about not having a smart phone and life without it. To help understand where I was, I had a Palm Treo 650 for two years and a BlackBerry Curve 8330 for another year. So, all told I had a smart phone glued to my body for about three years. Help desk tickets came to my phone, personal and business e-mail, calls, network monitoring pages, news, etc. I was one connected (to work) kinda guy. I thought that I enjoyed all of this connectedness and that it made me a more valuable employee. Or, was that simply a perception of mine?

As a former boss of mine once said "perception is everything." How true that statement has turned out to be over the years. My perception of being glued to work 24x7 I thought would make me more productive and a more valuable employee. I could respond quickly to questions, emergency's, I could even secure shell into some of the Linux boxes from my smart phone. Working while enjoying a high school football game.

I kept up on personal stuff too. Twitter, Facebook, gmail, news, stocks, the list goes on. The joys of a smart phone.

That perception has been shattered now that I don't have that stupid little thing. As one moves through life and the changes that come with it, we should learn from everything. Life is a fluid one-way event that we have to enjoy along the way. Some areas of the path are rocky, dark and sometimes downright ugly, but I digress.

Liberation should be the word of the week. The realization hit me that all of that connectedness didn't help me out at all. It added to my stress level and frankly people can wait if they send me an e-mail. My BlackBerry isn't chiming each time a new e-mail hits my inbox. In fact, Outlook is closed as I write this post. I've resolved to only open Outlook twice a day, respond to e-mails and keep it close the rest of the time. If there is an emergency people can call my cell phone.

So, yes, perception is everything, but we should closely evaluate our own perceptions of ourselves and what we need and what causes stress. I'm feeling more productive and I have fewer interruptions.

Liberation.

Life with and without a smartphone

I'll have to admit that I do miss my Blackberry Curve. I had the phone for over a year and they are very addictive. Work was paying for it, they bought the phone too. But, budget cutbacks happen. The business has to survive and so what to do?

Cutting back is not always fun. It's easy to point to someone else and say "you don't need that" but I'm like everyone else and I don't like it when it happens to me. But I'm flexable and so what to do?

I enjoyed the calendaring aspects of the phone to keep me on track and at meetings. The integration with Exchange 2007 was great. As I like to point out, it was like having Outlook in my pocket. Contacts, tasks and e-mail were also super handy to have right there.

It's been nearly a month without the phone and the world has not come to an end. My wife is not dissapointed that I'm not checking the dumb thing every few minutes and I'm liking not being bothered with all of the e-mail that comes in.

But, I do miss the calendar. I've started moving a lot of my personal items, e-mail, contacts and calendar off to the cloud, namely Google. Now, I have Google voice and I'm looking forward to using that service. Google calendar offers the ability to SMS text my phone when I have an appointment. This will help me with my temporal issues and keep me at some of my personal meetings. Work, well Outlook is more than happy to bug me.

Thank goodness for the cloud and even with a "dumb" phone, I'm still able to keep my meetings and stay productive.

But, should I learn programming?

On thing I have found over the years as a senior systems administrator is that my programming abilities have always come in handy. Many times these two skills seem to be very different..and they are. Programming is taking a problem, find out how to solve it and telling a computer how to do the task. Systems administration is taking an existing working system and implementing it, tuning it and making it run and finally upgrading and decommissioning the system.

Two very different skill sets.

But, should I learn programming? The simple answer is yes. Maybe being a "code jock" is in your cards, but having it in the Swiss Amy Knife set of tools will make you a better systems administrator and a more valuable employee.

Another good skill is database work. Why? Well, what is Active Directory? It's a database built on the Jet database engine from the boys in Redmond.

Some good examples from work. Account creation. Who likes account creation, especially when you have a hundred or more to create in a short time. You have to reconcile and figure out what needs to be created, and then do the work. Not fun.

Ah, write a program to solve the problem. Taking my database skills I was able to interrogate our Administrative Computing system (CARS, or CX from Jenzabar) to come up with a list of students. I then interrogated Active Directory to see who has accounts. A little reconciliation and *poof* I have a valid list of accounts to be created. Now, how to create those accounts? I've been programming in PHP as of late because it's simple, robust and effective. But PHP can't interface with PowerShell directly. So, my PHP programs generates a .ps1 script that I then execute on the Exchange 2007 server to create the accounts. Oh, but wait there's more. We hand out a sheet for all of the students to have with their account information. Being an old Adobe PostScript developer I then generate a .PDF file with the information sheets. Print that out and the work is done.

Now, generating accounts is a five step process that takes about 5-10 minutes to build out 100+ accounts that are accurate, mailbox enabled with hand outs. Not too bad.

If I didn't have programming and database skills this task would be far more complicated then it needs to be. So, be a good sysadmin and learn some coding skills.

Did I mention that coding helps monitor the network? No? Well, we use Nagios to monitor the network. When something fails and we were not monitoring for it, you guessed it, I write a script to do so. Sometimes it's C, C++, PERL, PHP, Bash, whatever it takes to get the job done. Now we are able to more quickly identify problems on the network before our customers notice.

Be a good sysadmin and learn some coding skills, it will pay off for both you and your customers.

Server changes -- why plan?

Well this week we had some interesting server issues to resolve. One of the guys decided to try out the VMWare converter and move a domain controller to a virtualized environment. All seemed well until Active Directory decided that it didn't want to replicate on that controller

The root cause of the problem was a USN rollback situation. Fixing the problem turned to to not be very nice. I had to forcibly remove Active Directory from the system, cleanup metadata on the other servers and then DC promo the box back into the domain. This all took the better part of a day to ensure that all is well and well done.

The bigger issue turned out to be Outlook Web Access. Through the process it totally messed up OWA on the system. We had a new VM setup to move it onto. This was the needed "nudge" to get the system moved. We had some problems with our wildcard certificate, which was mostly my problem of not installing it properly.

So after several days of Server 2003 and Exchange 2007 work all is well with Active Directory and Exchange's OWA front end.

Why plan? Well, a little research would have shown that there are steps and procedures to make sure that the move was successful. I don't like down time on any system. I have learned over the years that most of the work that I've done on servers has been done by someone else, so learn from their mistakes. This is a lesson that I've learned from being a pilot. We read accident reports and learn what our fellow (and less fortunate) pilots have done. Learn from their mistakes so we don't make the same blunders. All accidents -- and server problems -- can be related to some chain of events. Break one of the links and the problem won't happen.

Coming from an aircraft manufacturing background, two primary things were emphasized: Project Management and Six Sigma. Sure, some could argue that Six Sigma doesn't apply to IT work, but I would argue that it does. Well written procedures will result in good results. Same with making parts. Also, Project Management will force one to think and to plan their work.

Think, plan, execute and learn from one's mistakes. This will result in better service for the customers and make you look like a hero.