Biggest Threats to a company that have nothing to do with security

• A manufacturing company, year 1997, a LAN Admin checks a box about authentication requirements.
  
  • End Result: All users attempting to log in are prompted repeatedly for additional credentials.
    
• An international law firm, year 2000, an engineer pushes a new group policy to all workstations.
  
  • End Result: 45 minutes after all of the domain controllers receive the policy and push to the workstations, all users are unable to use any software listed under c:\program files\*.*
    
• A large telecommunications company, year 2005, an engineer pushes a new antivirus policy to all windows servers in the DMZ.
  
  • End Result: All of the servers slow to a crawl as all files opened or created are scanned in real-time.
    
• An internet bank, year 2010, a contractor pushes a software security patch to all systems.
  
  • End Result: The software is installed on 3 different Windows server versions, in some cases, rendering mission critical legacy systems in an unstable state.
    

While none of these individual incidents may seem to present a huge threat, the attitude that permits them to occur does. Without processes and policies that discourage unscheduled or "un-approved" actions, people WILL do unauthorized work and at some point, there will be negative ramifications.

What is the cost of unauthorized work in downtime? While the answer to that question depends upon the environment, any cost is too much. Change control while different from ITIL's change management at a minimum controls the changes in an environment. The first aspect of change that has to be managed are the attitudes of the people involved. Unfortunately, technologist too often forget that they are working in a live environment with impact to end users who are then unable to do their jobs and make money for the company they work for. That's because impact is frequently undocumented.

Why do we need change control?

Millions of dollars of lost business opportunity can be contributed to environmental changes, whether they are technology changes or others. Being able to predict availability and stability can contribute to a stronger bottom line. These reasons alone are sufficient to warrant a change control policy. In a regulated or publicly traded business, changes that cause issues can also have regulatory or reputational as well as monetary impact.

How do you control unauthorized changes?

Having policies and procedures that reflect an attitude of intolerance toward unauthorized work is a beginning. The initial best step is to write a policy basing the documentation requirements upon risk to the environment. For example, making a change to one end user's desktop "should" be fairly non-impacting to the organization overall. Document what work is considered sufficiently low risk as to be permitted with minimal paperwork or documentation. That type work could generally be done with an email request or if the corporation has a ticket system, with a ticket request. Enforcing a change control policy will not make you popular but it will make your environment far more stable. The policy has to have support and be enforced for all technology groups. The policy has to have "bite", in other words, the terms for non-compliance must be severe up to and including termination.

Include in the policy:

• A methodology for rating the risks related to the changes
  
• Approvals needed
  
• Change exception process
  
• Emergency change process/approvals
  
• Testing requirements
  
• Rollback process
  
• Communication plan
  
• How to document implementation
  

Steps to take after you have a policy in place include:

• Train the technologists,
  
• Train the business (not to the same depth as technology but they need to be aware that as stakeholders they will need to approve changes),
  
• Setup meetings to discuss changes whether on a daily, weekly or monthly basis depending upon the scope and number of changes in your environment,
  
• Review high impact or risk changes on a regular basis,
  
• Review the change outcomes and improve upon success rates. Measure, document and report.
  

Of course the best way to control unauthorized changes is to have systems in place that prevent unauthorized changes and in the event there is one, roll it back. These systems can be cost prohibitive to a small or medium business and in that level of environment be more bureaucratic than necessary. Maintain the stance that the systems needs be no more complex than the risk profile of your business. If you are in a non-regulated business, the change approval process may not need to be as stringent as it would in a regulated business.

What happened in the situations documented above?

LAN Admin @ Manufacturing Co –100 employee impact - there was no central helpdesk so pockets of employees were complaining to each other before someone called the LAN Admin. Approx 2.5 hrs downtime.

GPO Push to desktops – 1500 employee impact - the helpdesk began getting calls and they notified the engineering team who scrutinized the change that had been made and realized the issue. Approx 3 hrs downtime.

Antivirus push to systems – 3000 servers received the push – operations engaged engineering when alerting realized a change had been made – the push occurred on a Saturday evening at 7pm. Engineering was able to engage the appropriate personnel and after investigating corrected the issue before customer impact. Clean up was completed at approximately 3 am. 0 downtime.

2010 software push – several of the systems had received the push but not attempted installation. Less than 10 systems ended up having to be rebuilt. Approx 5 hrs downtime.

Next steps:

See next blog. We'll progress through other areas that can contribute to instabilities and lapses in any business.