AI in control

It is not long ago that digital control systems began appearing on overhead cranes. They offered such things as anti-sway control, collision prevention and true vertical lifting of loads – all of them huge aids to safety, and all, until then, the domain of skilled and experienced operators with many years of experience.

Today, anti-sway and the like are standard. It would be an unusual installation that did not have them. Sensors and digital monitoring provide millions of bytes of data on usage and wear and tear on the crane, and software analyses it for predictive maintenance that will avoid unexpected and costly downtime from failures and breakdowns.

With cameras added to give remote vision, laser sensors to give accurate measures of distance, and Wi-Fi for remote monitoring and operation, many cranes have become all but automatic.

All this a few years back sounded revolutionary; today, it is in danger of becoming yesterday’s technology. The digital world, in case you hadn’t noticed, is advancing at breakneck speed. It no longer relies on human programmers writing code for each task. Artificial intelligence and deep learning mean that the computer now teaches itself. The latest development making headlines is the AI Chatbot GPT, released earlier this year to the public for free, initially at least.

It teaches itself, or learns, by example – the examples being the billions of words and sentences that have appeared on the World Wide Web up until 2021. It analyses them statistically, to see which ones go frequently together – parts of speech such as nouns and verbs, words linked through topic such as ‘crane’ and ‘hook.’ Its backers claim it can write essays or magazine articles – or, if you prefer, Shakespearean sonnets – on any topic, in any style, instantly and on demand, which for the (human, I promise) journalist writing this article is more than a little worrying. Teachers fear their students can now just delegate their homework to their laptop-installed Chat GTP and no one will be able to tell if it the essay was written by student or computer. If AI and deep learning can do that, why on earth cannot it also control a crane?

The short answer is that it can. According to one, highly speculative, analysis, 65% of crane operator jobs will be replaced by robots. The job is ranked 363 out of 702 for replacement (source: www.replacedbyrobot.info). Another analysis (willrobotstakemyjob.com) puts what it calls the “automation risk level” of crane operators at 72%, with a 47% risk of the job being done by computer within the next two decades.

AUTONOMY PROMISE

Automatic cranes, as we have seen, are here already. Automatic, though, is not quite the same as autonomous, and autonomy is what AI promises. The automatic cranes does what it is programmed to do, repetitively, and that is all it does. Faced with something unexpected, new, unprogrammed, it would, one hopes, automatically shut down, for safety, and then a human operator must step in to decide and implement its most appropriate next action.

An autonomous crane, on the other hand, can cope with the unexpected. It will decide, by itself, what to do to best carry out a task, and then do it. No human will have programmed it to carry out those actions; it will have taught and learned for itself. It will be a change as great as the digital revolution that we have had already and will make that revolution seem a mere blip. And the autonomous crane may be already with us.

The gap between automatic and autonomous is huge. The classic example is the driverless car. It has been promised for decades, and not only by Elon Musk; for years, it seems, it has been ‘just around the corner’ – but not quite yet arrived. But driverless cars are now in limited operation as taxis, in a small number of US cities (generally, but not always, with human sitting in the driver’s seat doing nothing but being there to override the car’s driving decisions if necessary).

An autonomous crane can be much simpler than an autonomous car. A car has to navigate in the real world, the complicated crazy world of unpredictable people, of other drivers each doing their own thing, of pedestrians or dogs or children with footballs who can step into the road without looking, of dark and rainy and foggy nights, all of which the car’s vision sensors must try to interpret and react to. An overhead gantry crane feeding waste into an incinerator in a waste-topower plant inhabits a far simpler world: humans are rigorously excluded from its pick-up area and there is nowhere the gantry or trolley can go except along their rails.

Docksides and container ports lie between the two environments: more controlled, more uniform, not as complex as a city but with much more complicated requirements than a simple A to B bunker-to-furnace movement for their cranes. They have been at the forefront of automated loading and unloading operations. Both dockyard gantry cranes that load and unload the vessels, and rubber-tyred gantries (RTGs) that move, stack and store the containers, are involved. Konecranes is a leader in that field, and Sami Terho is the company’s senior specialist for crane intelligence.

“This does seem to be the time when AI and cranes are starting to take off,” he says. “As with autonomous cars, the autonomous crane is close. Progress in the car industry is very fast: the volumes are so huge and there is so much research going on in that area.” And there is, he says, some crossover: “In general, the kind of the techniques you are seeing in cars are a bit similar to what we are using in cranes also. The robot operating system (ROS) or its latest version, ROS2, that are used quite a lot with the autonomous cars have interesting aspects also for the crane domain – for example, in perception of the environment.”

The old ways of control – for antisway, for example – he calls “machine control”. It involved a mathematician encoding equations from physics that linked the gravitational constant, the length of a pendulum and its time of swing with accelerations of the trolley and pay-out speeds of the rope and hook. All of those would be combined into an algorithm and that was what the computer used. Machine deep learning needs none of that. It does not need to understand physical laws. It only needs trial and error and experience or data.

A mathematical model can only predict what it is designed to predict. It cannot predict that a careless shopfloor worker is about to walk into your nonswaying load.

But AI can recognise a human being and the direction that they are moving in: “They have trained autonomous cars with millions of example images of a human from the point of view of a car. The car has been shown what a walking human looks like, or a sitting human, or a human lying down on the ground. And then when the car is driving it can detect that and work out, ‘OK, now there’s a human in the road, so let’s stop. Or let’s try to go past it safely.’” Apply that to cranes and there is a clear safety benefit.

“In crane control systems there are so many levels, and it is difficult to replace all of them with machine learning or AI,” says Terho. “The low-level systems, such as anti-sway, I think will still be implemented with traditional control engineering methods. Warehousing also works quite well with traditional methods already. I think it is unlikely that there will be deep learning on that level. I think that mainly AI and deep learning could be used in the higher-level systems, on top of machine control. Perhaps for building additional features and functionality – observing the environment and recognising what is there, for example, and preparation for different situations, warning systems and that kind of thing. And, of course, they will link into other machines in the plant as part of Industry 4.0 to move the whole process, not just the lifting, towards autonomy. The main thing is that AI systems are more about complementing the traditional approaches rather than replacing them.

“To make a crane fully autonomous, the main requirement is safety,” says Terho. “You must make sure that it doesn’t hurt humans directly or indirectly. To replicate the kind of understanding that a human has, and the capabilities of a human for making the system safe, is a really complex task and you need to make lots of extra effort to make sure that the resulting system is safe.”

Artificial intelligence explained

Rafiq Swash is CEO of specialist AI control company Aidrivers, based in Uxbridge, London. Here, he explains some of the problems, principles and solutions behind AI and machine learning in controlling cranes.

“On any crane you suffer from sway. One way to tackle it is to remove it altogether. That is the old, digital-mathematical way of doing it. The crane starts with gentle movements – light accelerations and decelerations of the trolley. It does indeed eliminate sway, but production suffers because the accelerations are slow.

“A more productive method is to’ handle’ the sway – turn it from a problem into an opportunity. Judge it right and near the end of its run the load might be swinging outwards, just over its setting-down point. So if you lower the load at just the right point in that final outward sway, you have put the load down in the right place, and done it faster that using the algorithm-based slow-acceleration method.

“This what the most  skilled crane operators do – perhaps knowingly, perhaps unconsciously. Human beings are, let’s face it, very clever. An AI crane does that also. It handles the sway rather than eliminates it. And productivity benefits: the method is much faster. And if the load sways a little on the way – so what? The human operator can judge whether that is a safety issue or not. Deep learning can do the same.

“Mathematical algorithms can model the physics of a crane lifting an object – but only up to a point. Consider, for example, the movement of, say, a real boat though a real sea. Projecting and modelling that is extremely difficult because the ocean goes on forever and has waves. Waves are part of the global weather system so are affected by the tiniest global changes – climate change, a storm the other side of the planet, even the proverbial butterfly flapping its wings and raising a storm – but people have for thousands of years been able to navigate a vessel into port. They do it not by understanding the mathematical and physical equations that govern wind and waves but by observing how their boat changes direction when they alter the angle of the rudder and sails.

“Similarly with cranes and hoists: all kinds of things affect them. A slight temperature difference changes the flexibility of the rope and also the density of the air and therefore the air-damping of sway. Each load will be of a slightly different weight and its centre of gravity will be slightly different. I am not claiming that it is impossible to model it all mathematically, but every situation is different, so for a truly autonomous crane one mathematical model will not fit all. It is not like, for example, an automatic siding door on a train, where the actions and end points are much more limited.  

“So the new way of crane control is cognitive. It tries all sorts of solutions while it is learning, and adopts the one that gives the best results. And it is always learning, and therefore improving itself.

“With conventional sway control… port cranes were [observed to be] handling 25 movements an hour. With conventional anti-sway disconnected, human operators were able to do many more (and we found that sometimes they did indeed disconnect the anti-sway, in order to be more productive). And the AI deep learning cranes similarly were much faster and more effective.

“With AI there is no need to pre-programme algorithms. We call the new way ‘cognitive control’. It takes, of course, greater computing power. You need to build some level of cognitivity into your crane (which is not the same, of course, as being conscious, or alive). It recognises the problem – how to use sway safely – and finds its own solution.

“Will it take jobs from people? No: it will change the jobs that people do. Every generation finds jobs that work for it. We have the huge benefit now that education is so much more accessible, to almost everyone. The current workforce will be the last that is willing to work in a tiny crane cabin all day, alone, at height, on a repetitive task. So we need to create new jobs for the new generation, the one that is currently growing up familiar with technology.

“Our first project for AI in cranes, and the one that sparked our interest in it, was for a waste disposal plant in China. The crane was moving domestic refuse, and the smell was terrible. Once we had installed AI no operator had to work in it.

“It will be ten to 15 years before this is everywhere. We need to find creative ways of using people’s skills and interests. But with education, those ways will exist.”

Andritz logyard crane in Finland

The world’s first autonomously operated logyard cranes have been successfully erected at Metsä Fibre’s bioproduct mill in Kemi, Finland. The 25t gantry cranes have been supplied by international technology group Andritz.

The project includes two cranes on a 540m-long runway; they will be servicing a logyard with a storage capacity of approximately 120,000m3. The cranes will unload approximately 7.6 million cubic metres of wood per year, shipped in on trucks and trains, and handle log storage. They will also feed the logs into the pulping process.

The cranes feature the latest in artificial intelligence. Compared to traditional log-handling solutions the AI will optimise log handling, minimise wood losses and secure environmentally friendly and cost-effective operation.

Planning of the new mill is based on a high level of environmental efficiency as well as efficiency in energy and materials used. The mill will not use any fossil fuels at all, and its electricity self-sufficiency rate will be 250%. The Kemi mill will produce 1.5 million tons of softwood and hardwood pulp a year. Start-up is scheduled for the third quarter of 2023.

Flexible robotics from Jaso Cranebot

Is it a crane? Is it a plane or a sanding machine? Is it an AI-controlled robot? Yes – it is all of those things. It is also the Cranebot, from Spanish crane makers Jaso.

The company describes it as a flexible robotic crane. It “sums the advantages of smart cranes and parallel cable robotics to respond to the automation of industrial processes” Jaso says. It has a gantry, like any other overhead crane, but from it hangs not a trolley and hook but a platform suspended from eight cables. The platform carries a machine-head, a planer or a sander for example. The cables are wound in or out, independently, computer-controlled via laptop by a CNC module; the platform tilts, moves and rises or falls accordingly, with six degrees of freedom, or even more if an additional robotic system is used on the platform. It can trace out the most complex of 3D shapes, forming and smoothing the surface of the workpiece below it. It has been used to shape and polish giant wind turbine blades, the shapes of which are critical, complex, and must be smooth. Naval construction and aeronautics are two of many other applications.

The platform can be fully controlled in position and orientation, with high precision and without oscillations in any direction and in any orientation. The Cranebot can be retrofitted to existing overhead cranes.