Why Your Software Pilots Fail: The Case for Executive Ownership

The pilot graveyard in this industry is not full of bad software. Some of it is, sure, but most of what is buried there worked well enough and simply never had anyone senior standing next to it while it was being judged. In this episode of Construction Conversations, Lars and I sat down with Elliot Christiansen, Senior Vice President of Operations at Cleveland Construction, who put seven systems into the field in eighteen months and came out the other side with teams asking him for the next one.

Elliot is on the operations side, not innovation, which matters. He came up as an APM, spent his first five to seven years as a project manager on site, and inherited the technology portfolio almost by accident in 2015 when his IT director pulled him into an office to look at something that integrated with Sage 300. That became a Procore pilot on his own job, an outlet mall in Columbus, and once you run the pilot you own the rollout, and once you own the rollout you own the training. It kept snowballing until it was just part of the job.

Here’s what stood out.

1. If your name is on the pitch, you don’t get to leave the room

Most executive involvement in a pilot ends at the approval and picks back up at the ninety day readout, which is a fine way to receive information you cannot evaluate. Elliot’s rule is the opposite, and his reasoning is almost boringly practical.

“If I’m gonna put my name behind it, then I’m gonna be involved in it throughout the rollout, throughout the pilot process and make sure that if our guys are saying it’s good or bad, I’m gonna make sure that I am right there with them.”

The point is not enthusiasm, it is diagnosis. If the team comes back and says the tool is bad, he wants to know whether it is actually bad or whether they never opened it, and from a conference room those two answers are indistinguishable. Being close to the work for as long as he was is what lets him tell the difference, and it is also what lets him filter the noise before it reaches the vendor, because as he put it, an RFI is processed the exact same way today as it was twenty years ago, it is just in a system now instead of a fax.

2. Put the person who will hate it on the pilot, on purpose

This is the sharpest idea in the episode and I have not heard anyone else say it out loud. Every Cleveland pilot gets seeded across multiple demographics, and two of the seats are deliberate.

“I’ll usually try to pick somebody that I know will be a detractor from it in the pilot, and then pick somebody that I know will be down the road be a champion for it.”

The champion goes in early so they are ahead of it and can train project teams later, which is the obvious half. The detractor is the half everyone gets wrong. He will list everything wrong with the product, in detail and with feeling, and that list goes back to the tech company while the pilot is still running and the roadmap is still soft. Most companies keep their skeptics as far from the pilot as they can, which is a reliable way to find out what those people think at renewal.

3. Self-reported adoption is the least reliable instrument on the jobsite

Elliot was on site recently and asked a superintendent how the new RFI tool was going. The answer sounded great. “He goes, I think we asked 12 questions in it the other day.” So he called the rep and pulled the usage for that job.

“That guy had asked zero questions.”

The twelve were real. They just belonged to everyone else on the team. It is a small story that quietly indicts a lot of pilot reporting, and Elliot’s response to it was structural rather than personal. He now makes a point of only working with partners who will hand over usage dashboards on a regular cadence, either available to him on demand or pushed from their back end, which is how a vendor actually helps a champion do the accountability work instead of leaving him to run on sentiment.

4. Straight line time savings sell themselves. Everything else needs an offset.

Some tools you can see on the demo. Submittals were that, and the reason is that we hand the least experienced person in the trailer a two thousand page spec book and a stack of product data that no two vendors format the same way, and the training program is largely getting yelled at for what not to do. After rollout, initial submissions started going in correctly more often than not, and an architect on that first job was impressed enough to recommend Cleveland to another owner. Elliot found that out from one of his own APMs, on a webinar, well after the fact.

Not everything draws that clean a line, and he is honest about which ones don’t.

“Superintendents are very skeptical people. They’re gonna ask a question and it’s gonna give them an answer and they’re still gonna check.”

Nothing is saved until the trust arrives. So OpenSpace went out with an offset instead of a promise. Keep taking progress photos, but you are no longer required to shoot a thousand pictures a week and tag every one with a description and a location, because the weekly walk is on the drawing. Then came the use cases, like pulling it up inside the sub meeting rather than adjourning everyone to go look at the wall.

5. Six platforms in eighteen months, and a speak-now meeting

The list is not small. Microsoft Project to P6, Procore to Autodesk, plus GCPay, OpenSpace, Trunk Tools and Breadcrumb, which touches essentially everyone on the operations side. Elliot had been the biggest Procore supporter in the company for ten years, so he was moving off a platform he had personally championed for a decade.

What they did before the Autodesk decision is the part worth stealing. Roughly thirty people sat through three demo calls, superintendents and estimators and project managers and back office accounting, anyone who would have to touch the system. Then an internal meeting, speak now or forever hold your peace, where Elliot stood up and explained why they were looking at all, which came down to cost, an audit and an overage fee with no credit coming back the following year when they were under. Five months to the day they will be off Procore completely, with small complaints and very little resistance.

The tell that any of it worked is not in a dashboard. It is a project team with eight months left on a job asking whether they can add the safety platform nobody was ever going to make them adopt.

Conclusion

Elliot says they are in the good for now stage on pilots, with one more he would like to run if he can get a couple of people on board. What comes after is the harder problem, which is pulling data out of all six of those systems into one warehouse so the compliance checking and the WIP reporting stop eating a week of somebody’s month.

Which is the part of this that keeps nagging at me. We spent a decade arguing about whether the tools were good enough, and Cleveland’s answer is that the tools were mostly fine and the variable was always who stayed in the room. The next round of value is sitting in what all six of those systems are quietly writing down, and that one will not be won by whoever buys the best product either.