Response Plan" in reaction to the IT Payroll Failure. This is a plan to prevent the same type of disasters going forward. Write your plan in the form of a memo/email to the company's leadership (also address any other relevant parties). Include each team member's email signature including where you are from or geographically located and, at your discretion, your current company and position. This is not required and is primarily so I can some sense of your backgrounds. You may want to solicit information from Professor Cumbie or even float an idea in an CNVS Inbox message or even setting up a call: voice, Zoom, or otherwise. Do not try to fix all the problems at once. Focus on one particular aspect. Think: what is the foremost step to take. Will Saturdays, SEPTEMBER 4rd Headline read: “$4B Company Can’t Pay Its nearly 4000 Workers!” That’s how things are looking right now. An IT Systems failure at PARTS CO. caused a PAYROLL RUN failure. Friday morning, data from the FINANCIAL SYSTEM printed zeros for all employee wages and hours worked, affecting thousands of hourly workers across the US and locations in North America. The good news is that the cause was found (whew, headline avoided!) and systems restored around midnight. The bad news is that · this was well after the 5PM payroll deadline and that workers got paid 1-3 days late, Wall Street analysts suspect severe cash flow issues, · audit red flags are raised all across the General Ledger, · employee morale is down, potential lawsuits from Union and labor law violations, and all of this on your first day as newly appointed VP of IT OPS. You just reluctantly accepted a promotion from Director of Midrange Technology Operations that morning from the CEO, Steve, after both the CIO and VP IT OPS were terminated. Your mission from the CEO? Do not detract any resource from PHOENIX (PHX): the company’s $20M over-budget and 2-year overdue project to catch the competition on e-commerce channel integration, personalization, recommendations, and mass-customization. Before you even could make it back to your office, the emails and calls started about the PAYROLL failure. RESPONSE The joint team of finance and IT personnel decide to go with a plan to pay employees based on the previous week’s PAYROLL RUN. It will generate a lot of corrective work (overpay, underpay) and likely trigger an audit but may stave off violations and lawsuits. It is also deemed best to the employees, some pay is better than none. Ann pulls last months employee pay data from the DB into a SS: the SSN are gibberish, the hours and wages are 0. Its displaying Kanji-like symbols. Just like that, the plan is out; there is not even historic data available to use for this pay period. At least this is an IT clue: one corrupted table field (SSN) doesn’t sound like a SAN failure. If the SAN were completely out, we’d expect nothing from the DB. You take action and ask for a list of all system changes in last 3 days making sure to hit up everyone related to the TIME app: DEVS, System Admins, NW PPL. Since our TICKET system isn’t uniformly used, do this via email. By 2PM Patty finds 27 changes in the last 3 days. John’s Tokenization and the vendor firmware SAN upgrade stand out as top suspects. Some had suspected the ERP MANU System since it is 20 years old, runs half of the business and is considered very fragile. · 7PM - The tokenization on the TIME APP is found to use a different character encoding set than that on the DB Server. · 9PM - The DB Server is updated with the character encoding and tests pull clean payroll data. · 11PM - SAN is Back up and running. All systems restored. SEV-1 concluded. The PAYROLL RUN is restarted and succeed. Data are uploaded to vendor for processing and employees are payed, 1-3 days later than expected. OUTCOME You handled this SEV-1 outage pretty well for your first day but the damage is done and the next SEV-1 outage is likely lurking just around the corner and PHX isn’t any closer to completion. You know what they say about the life in IT OPS? You are always one career ending outage away from being terminated. The question is now: what do you supposed to do? The company is blaming IT again and the entire IT branch is looking to you for guidance. Steve wants a memo on his desk ASAP. His words: "How are we going to right this ship?" INSTRUCTIONS Research the supporting case materials to familiarized yourself with the structure, culture, and processes at parts. Look for guidance from the supporting IT management documents. , Prepare a memo for the CEO outlining your initial response plan and including the following: (1) define the problem at PARTS: 50% of the solution is defining the problem (2) formulate an Initial Response Plan: What needs to happen, when, by whom, & why? (3) what more information do you require to define the problem or to formulate your plan? Prepare questions and request to ask the Professor for additional PARTS or related information. 。 LATER THAT NIGHT You have a premonition of things to come in the next month: · September 4 - Internal Audit findings of insufficient IT controls and significant material weaknesses · September 9 - Credit Card Processing Failure where no retail stores can process transactions · September 12 - Phoenix Deployment Disaster that exposes customer financial data, mixes up and loses transactions, overcharges and undercharges clients. · September 18 - Invoice Incident where no large clients have been billed for five days, ceasing cash flow, causing the sales team to miss their quotas and accelerators, and leading to a $50M shortfall on projected earnings for the quarter · September 26 - External Audit scheduled SAN UPGRADE A vendor field engineer was on site to install a firmware upgrade on the Enterprise SAN, just 4 hours before the PAYROLL RUN on Thursday. Around 7PM it was taking longer than expected and Brent came on to help. They were many version behind and the vendor never tested the upgrade path. They’d been meaning to upgrade the SAN firmware for years but never go around to it because they couldn’t get big enough of a maintenance window to take down several servers. The performance had been degrading with critical apps being impacted. After a reboot, all the tests failed. Just 15min later the payroll run failed. They attempted to rollback the change but the SAN bricked. The SAN is a million-dollar piece of equipment with irreplaceable corporate data. PII TOKENIZATION ISSUE. In one month, the PCI (Payment Card Industry) auditors are visiting. The Personally Identifiable Information (PII) for employees is stored in plain-text in organizational databases used in the Timekeeping APP (TIME) among others. European and US State Laws prohibit storing non-encrypted data in this manner; a possible audit implication. 。 John, the CISO, has found a product to tokenize this information but the deployment is already 1-year overdue. There is a 4-month queue to get in the standard Ticketing System (TICKET) for the work order. There is not a environment to test these changes before they are put into production. Max, a DEV maintaining TIME was asked by the CISO to get this security feature installed before Max started his vacation on Friday. He was having CXN issues with the plants and called up Brent with a question re: DB table structures last week. Brent gave a quick answer while he was in the middle of testing PHX VMs. PAYROLL RUN PROCESS *A weekly recurring process starting each Thursday night. Calculate Pay Subprocess · Input: hours worked for hourly workers from TIMEKEEPING APP from various divisions and plants and retail stores across many states. · Input: salaries and wages from HR DATABASE as well as state tax tables, employee withholdings, garnishments, vacation and sick leave accrual, employee benefits, and so forth. · Process: Export data onto spreadsheets and compile into a single PAYROLL SS · Output: gross pay, deductions, net pay and other payroll information per employee in CSV format. Resolution Subprocess · Input: summation of payroll and related account data from payroll spreadsheet · Process: verify totals from summation to detailed numbers (manual comparison) · Process: correct errors manually using an IT created tool to FIX errors. This is a “clunky, error-prone process” · Output: verified total numbers SS · Output: verified detail numbers SS Report to General Ledger · Input: verified total numbers SS · Process: upload to FINANCIAL APP · Output: entry recorded on General Ledger Upload to Payroll Processor · Input: verified detail numbers SS · Process: upload to payroll processor VENDOR online system · Output: employee payments via VENDOR I've listed a few things below that I think are possible solutions with the current payroll system that we can possibly address in the memo. I started writing something up for the mandatory annual IT audit in the attached file in case we decided to include that solution. Let me know what thoughts you ladies have and how you would like to split the assignment. Have a great day everyone!
• mandatory annual IT audit
• perform all major updates in the test environment before planned go-live dates to ensure changes are working as expected
• require all users to utilize Ticket system for any work performed by IT so changes are easily tracked and monitored
• Routine payroll audits both internally and externally
• Eliminate organizational inconsistency – restructure HR, payroll, and data management functions if needed
• Consider storing historical payroll data in a location with restricted access so that it can be easily accessed in the event issues are encountered with the payroll system in the future