The Central Processing Unit (CPU) is the brains of the computer. The purpose of the CPU is to process data and instructions. Without the CPU, a computer would not be able to compute.
If a device has a button it probably has a CPU (and is therefore a computer). They carry out tasks very quickly: the CPU in your smartphone can carry out over 1 billion calculations a second.
Computers like Colossus, while electrical, and stored data electrically, were programmed mechanically using wires and switches. If you wanted to change the program being run on the computer, it would have to be re-wired. Imagine having to rewire a computer to swap from Word to Chrome (or from Fortnite to Minecraft)!