By the year 2050, our planet will need to feed ten billion people. We can’t expand the earth to create more agricultural land, so the solution to growing more food is to make agriculture more productive and less resource-dependent. In other words, there is no room for crop losses or resource waste. Bayer is using Amazon SageMaker to help eliminate losses from happening in fields around the world.

Households contribute to food loss by discarding food such as kitchen waste or leftover cooked meals. However, the vast majority of food loss in many countries is actually from crops that “die on the vine” in one form or another—from pests, diseases, weeds, or poor nutrition in the soil. The Climate Corporation—a Bayer subsidiary—provides digital farming offerings that help resolve these challenges.

The Climate Corporation’s solutions include automatic recording of data from tractors and satellite-enabled field-health maps. By delivering these services and others to thousands of farmers globally, The Climate Corporation enables farmers to keep their land healthy and fertile.

The team is also working on an upcoming service called FieldCatcher that enables farmers to use smartphone images to identify weeds, pests, and diseases. “By using image recognition, we provide farmers with access to a virtual agronomist that helps with the often difficult task to identify the cause of crop issues. This empowers farmers who don’t have access to advice, as well as enable all farmers to more efficiently capture and share field observations,” said Matthias Tempel, Proximal Sensing Lead at The Climate Corporation.

FieldCatcher uses image recognition models trained with Amazon SageMaker, then optimizes them for mobile phones with Amazon SageMaker Neo. With this setup, the farmers are able to use the model and get instant results even without internet access (as many fields lack connectivity). Using Amazon SageMaker helps FieldCatcher to identify the cause of the problem with confidence, which is critical to providing farmers with the right remediation guidance. In many cases, acting immediately and being certain about an issue makes a huge difference for fields’ yields and farmers’ success.

To power the FieldCatcher solution, Bayer collects images—seeking a wide variety as well as a high quantity to create training data that includes various environments, growth stages, weather conditions, and levels of daylight. Each photo is uploaded from a smartphone and eventually becomes part of the ongoing library that makes the recognition better and better. The figure below depicts the journey of each image and its metadata.

Specifically, the process starts with ingestion to Amazon Cognito, which protects uploads to the Amazon API Gateway and Amazon Simple Storage Service (Amazon S3). The serverless architecture—chosen because it is more scalable and easier to maintain than any alternative—relies on AWS Lambda to execute its steps and finally move the received data into a data lake.

Multiple AWS services work in concert to support the data lake. In addition to Amazon S3 for image storing, Amazon DynamoDB stores the metadata, as features of the image such as location and date taken are important for searchability later on. Amazon Elasticsearch Service (Amazon ES) powers the indexing and querying of this metadata.

The engineering team appreciates that this set of services does not require a data schema to be defined upfront, enabling many different possible use cases for images to be collected in the FieldCatcher application. Another benefit is that the data lake queries allow questions as different as “search for all images taken in Germany with an image resolution larger than 800×600 pixels” or “search for all images of diseases in winter wheat.”

For machine learning (ML) model development, training, and inferencing, the team relies on Amazon SageMaker. Specifically, Amazon SageMaker’s built-in Jupyter notebooks are the central workspace for developing ML models as well as the corresponding ML algorithms. Developers also use GitLab for source code management and GitLab-CI for automated tasks.

AWS Step Functions are the final piece, used to support the full roundtrip of preprocessing images from the data lake, automated training of ML models, and finally inference. Using these services, Bayer’s developers can operate with confidence in the infrastructure and can focus on the ML models.

The Bayer team members, as longstanding AWS users, are familiar with the power of ML to solve problems that would otherwise be exceedingly complex for humans to tackle. The company previously developed an AWS based data-collection and analysis platform that leverages AWS IoT and sensors in the harvest fields to power real-time decision-making with information fed to mobile devices.

Their choice to expand their offerings to include the new FieldCatcher application was driven by the positive feedback from some of these other services. Giuseppe La Tona, Enterprise Solution Architect at The Climate Corporation described, “We used to make this type of service fully ourselves, but it was an enormous amount of work to do and maintain. We realized that, with Amazon SageMaker, the solution was infinitely easier, so we started implementing it and have never looked back.”

At the moment, FieldCatcher is used internally in over 20 countries around the world. The next step is expanding what it can offer farmers. Right now, its main use is for weed, disease, or pest detection. The Climate Corporation is exploring additional ML-powered solutions as broad as predicting harvest quality with images and drone-based crop protection on an individual plant level. 

Going forward, the team plans to use Amazon SageMaker for all their ML work, as it has been so powerful and saved them so much time. In fact, the team’s entire workflow uses only AWS for ML. Alexander Roth Cloud Architect at Bayer, explained, “With machine learning on AWS, the huge impact we’ve seen is that the whole pipeline runs smoothly and we’re able to reduce errors.”

With these solutions in place and constantly improving (as is inherent to ML), Bayer and The Climate Corporation see themselves as pioneering the sustainable agriculture of the future. Their hope is that this effort and others it inspires will make it possible to support our growing population for years to come.

About the Author

Marisa Messina is on the AWS ML marketing team, where her job includes identifying the most innovative AWS-using customers and showcasing their inspiring stories. Prior to AWS, she worked on consumer-facing hardware and then university-facing cloud offerings at Microsoft. Outside of work, she enjoys exploring the Pacific Northwest hiking trails, cooking without recipes, and dancing in the rain.