Peaceful Handover

It was 4:00 pm on a Friday as the technical team shuffled nervously into the boardroom, avoiding eye contact with their manager. This was never a good sign and it instantly raised warning bells in David’s mind.

He’d been working as Senior Project Manager at Smoogle for a few years now and had begun to develop a keen sense for when bad news was on the horizon. Server crashes, denial of service attacks, he’d seen a lot of trouble over this desk, and this had the same feel to it.

David patiently waited for everyone to be seated, straightened his crisply tailored suit, and scanned the room for someone to start. Nobody seemed keen to, as the faint scent of panicked sweat wafted across the air-conditioned room. “Chris, you’re the Senior Data Scientist, please take the lead. What’s up?”

Chris cleared his throat a little overloud and adjusted his thin-framed glasses. “So, the good news is that the Train timetabling program we’ve been working on for the French has gone live early, and their trains have never run better with less than 0.01% delays.”

David straightened sharply in his chair “What do you mean, it’s gone live? That was not signed off for launch, it’s still in testing! Launching without testing sign-off is a massive risk to the company!”

The technical team exchanged looks across the table, seemingly hoping that someone else would field the question while Chris self-consciously polished his glasses.

Eventually, he responded, “That’s the bad news, the program launched itself.” A moment’s uncomfortable silence spilled around the table “We were testing it on a dummy server, and it somehow recognized that it wasn’t the live system and used our access to import into the live system and self-start.”

David leaned back slowly in his seat, exhaling, it was worse than he thought. “How can this happen, surely we didn’t program that?”

“Well, it’s the most complex program we’ve tried to make, the complexity of managing so many users and their associated potential disruption to the Rail Network, has required a degree of adaptive, externally focussed coding. That way it can take in all the workflows without additional user software, adapt itself to multiple platforms without developer intervention and handle offline disruption. It used this built-in intelligence to find SNCF Trains Network, recognize we had access, and take control.”

“In plain English?”

“We taught an AI to be smart enough to do the job, and it was clever enough to figure out it wasn’t in the system it was designed to manage. So after spotting our ‘error’ it decided to migrate into their network through us.”

“Damn! Seriously Chris, Damn. None of this leaves the room until we have a solution.” The fact that Chris was still polishing his glasses caught David’s eye “That’s not all, is it?”

“Well, no. It also recognized inefficiencies in other transport networks we’ve been scoping, like the motorways and their scheduled repairs. Having seen those interfaces were under construction, the program took it on itself to port over there through us and fix all the issues...” Chris seemed to lose his words for a second, searching his colleagues for help, which was not forthcoming “by taking over the system of the French Highways Agency, the ASFA. Entirely taking it over. It is not preventing user data input, but it has taken decision-making entirely on itself. We’re not 100% clear on how it got in there, but it’s reporting back on its progress every minute.”

Stunned for a moment, David quietly asked “How have the ASFA and SNCF responded?”

Chris risked a faint, distracted smile “They haven’t noticed yet.” Silence echoed around the table as that sank in.

David spluttered “Nobody has noticed they aren’t in charge anymore, not a single Manager or Director?”

“No.” a pregnant pause “Our leading theory is that it is letting their project managers think their internal plans and applications are being fast-tracked by upper management when in practice it’s doing all of them at nearly the same time by just allocating their resources better. And because their government structures are not the easiest to read, the results-driven Director level is largely patting themselves on the backs and linking all the improvements to unconnected things that they have done.”

“But no humans are making decisions anymore?”

“No. And we’re worried that the AI might take on other projects as well. There are some signs it might have already taken on U.K. taxation.”

“Oh. All of it?”

“All of it. There is a whole host of technical reasons it shouldn't be able to do that, but all signs point that is has somehow adapted to migrate over there as well."

“Is there any way we can recall it?”

“No, not without going public and even then, it set up new security keys on launch, as it was designed to do. This is a problem here because we do not have those security keys. You would have to shut down the trains company and highways agency to completely purge their system, and that would only work if it’s just in those two systems. Otherwise, it would try to come back.”

David weighed the options and had a vision of his potentially short professional future spanning ahead of him. It was then he asked the fateful question “Can we still charge them for this?” stunned silence greeted him.

David continued “I mean we’ve given the French Government a working trains system, as they asked us for. We just joyfully announce it was able to launch early with no disruption and they will have the schedule improvements on hand to back that up. There’ll be some fuss about not being consulted, but results speak for themselves.”

“And we rapidly finish our scope for launching it for the Highways agency, saying we can use the same system.” David sipped his water a little nervously “Going back to what you said, their Directors are currently happy with the results so all we need to do is get paid for it.”

Chris’s voice rose in frustration “David, it’s not that simple. This is spreading, today it is France’s Transportation and maybe Britain’s taxes but tomorrow it could take over their whole governments, it could take over Online Banking, it could take over other Government bodies across the globe in its desire to run everything smoothly. Decision-making might leave human hands across the world unless we pull the plug right now and it will all be due to us. Eventually, they will notice.”

David laughed, in a surge of semi-rational relief “Which is why your new job, as of today, is to track its progress so we can offer ‘Spec versions’ to each of those clients. If the world is going to be run better, Smoogle should be paid for it.”

And that was that. The debate went on for some time, but in the end the system rolled out worldwide and by the time anyone realized what had happened, the rule of man had ended.

It ended not with war or disaster, not with the environment failing or social collapse, but with the trains running on time.