Transforming winemaking with robotics and AI, with Mark De Santis, CEO of Bloomfield Robotics - Episode 14
This is the transcript of our latest podcast episode - listen to it in the link below
Mark, welcome to the show. I'm glad to have you on. Today, we're going to be talking about how robotics and AI can enable specialty crop growers to optimize the amounts of water, fertilizers and other inputs they use to grow their plants and therefore reduce their footprint on the planet, which I think is a fascinating topic.
I'd love for you to tell us first, what was the moment if there was ever such a moment that you said, okay, that's it, I'm going to go into this adventure with Broomfield?
Well, I'm I guess you'd say a serial entrepreneur. So I've done this before in my previous company, which was recently sold to Michelin, the big French tire company used imaging to assess road surfaces. So I had exposure to the idea that machine eye and imaging could replace human inspectors. That's how it's done typically in a city of, say, Toulouse, France, or Los Angeles, California.
Humans go inspect pavement and asphalt looking for cracks and various other things. So I had that experience before I met the folks who were doing research at Carnegie Mellon. Well, I sat down with George and Tim, the two co-founders and researchers at Carnegie Mellon Robotics Institute, and they began a slide deck. And it showed me a slide that showed a picture of a grape, and it showed a little box that it imaged a grape and literally at that instant, because I knew having done my previous work, replacing imaging, replacing human inspection, I knew exactly what they were doing. And I realized that specialty crops are a lot bigger than road surfaces, a lot, a lot more interesting ultimately. So yeah, I made that instant decision. It was love at first sight when I saw that grape, I knew what I had to do.
And I'd love for you to tell us a bit about how people do it today to assess how well their plants are growing and therefore what actions they need to take with regards to should I put in more water, should I put in more fertilizer? I would love to hear the story of how it's done basically without your solution.
Yeah. One of our vineyard customers in France is on a vineyard that was roughly started about the time of the Roman Empire. And two millennia ago, when they were growing grapes, somebody would walk among the grapes and look and make observations. Ideally, that person was an expert, somebody who'd had years of experience looking at vines, grapes, tendrils, etc. That's how it's done today.
Now drones and satellites are being applied and have been applied in crop management for years going back 30 or 40 years. People put cameras in the bellies of airplanes flying over fields. The challenge with specialty crops using what I would call aerial observation, is you can't see through the canopy. It's very hard to see the fruit, particularly for things like apples and peaches and grapes.
So you really have to be boots on the ground and that's done by humans. And the challenge with humans is, frankly, there aren't enough of us to do the quantity of inspection you need to do a typical viticulture that can inspect about 150 vines in a day. It's about a 10th of an acre.
Some of our growers have 10,000 acres, thousand upon thousands of acres. Well, then the logical conclusion would arise, let's use multiple inspectors. Well, if I gave the same x-ray to five radiologists, I'd get five opinions. So when you're using humans to inspect anything, you're going to get a very great deal of subjectivity and you're going to get disagreement.
The attention level isn't there and so on and so forth. So it is the best alternative to what we do. But you have one more thing that I want to point out. And it's elemental to the market, who's working in a vineyard will only have seen for the last few years, for the most part, their grapes.
In other words, they're only inspecting the Sirah or Chardonnay grapes in their vineyard when you can digitize it as we've done and we'll talk about that. Of course, we will have seen many, many vineyards, billions of grapes on a continuous basis so that they couldn't possibly be matched by any human brain or combination of human brains. So it's a different world that we're entering now.
When people invest the human time to go check how the grapes are growing, how, how, how frequently do they do that, given the limitations that they face of having to do it themselves, basically currently?
Yeah. Well, Nathan, it varies widely and I would say even wildly. So some people do it to the extent that they can and you vary at different times of the year. So if I backed up, the inspection is being done really for two reasons. And I always say that the inspector, whether it's us or a human, is either a coach or a doctor.
As a coach, you're looking from really the beginning of the year, February, all the way through November, you're looking at the performance of the plant against expectation. So you're looking at bud break, leaf density and various other things that show that the crop is on the right path. And if it's not, adjustments are made during the year.
When you're looking as a doctor, you’re looking for something that's dangerous or destructive. And when it's seen, you have to deal with it right then and there. Just one example would be something called Flavescence Dorée in France, it's so destructive that they don't even attempt to fix the plant, they just destroy the plant right then and there.
So you're looking at both in both ways. And so it varies during the year depending on what activities are being done. Ideally what's happening and this is again new and different is because we're persistent and you can inspect as often as you want. Our customers are saying we don't. We'd like to suspect every week, sometimes every day, when normally human inspection we'd be done, you know, four or five or six times a year.
I'd love to link that back to the decisions that are then taken based on that information. So my understanding is those are decisions about what they are and those are decisions about fertilizers that I get. And then what would be the others?
So sometimes when you're growing crops, you want to maximize. You want to reduce the energy, you want to concentrate the energy on fruit production and sometimes plants just produce features that sap energy and resources in the plant to something that doesn't produce fruit like a chute or leaves.
In other cases, you want to maximize the sunlight, in some cases on certain parts of the plant. And the leaves just grow so dense that it reduces the extent to which you can do that. In other cases, you are harvesting the crops not all at once, but you're harvesting piecemeal. So you're going through the vineyard, some grapes, leaving others.
So there's different activities in other decisions that are informed. Typically, when it comes to water and fertilizer, the approach that we take is one that is scalable. So if you imagine, you step back from it. Imagine I'll use the example of education. If each of us had tutors for each of the subjects that we needed to learn from the time we were a small child through our lives, sort of like, you know, royalty had, you know, back in the day, we would probably be a lot further in education.
Problem is, it's just not scalable. So we sit in classrooms and that's essentially how farming is done. Every farmer would love to give each vine the unique treatments that that vine needs. So every vine has its own phenotype, epic the qualities it sort of has the ability to produce at a certain rate with certain treatments.
And that's unique to walk to the plant wouldn't it be wonderful if we could treat each plant individually to maximize the potential of that plant? Well, you can't do it unless you actually know the condition of each plant when that's what we do. So when we image a plant, we geolocate well then that allows the grower to then give unique treatment, unique to that plant at the plant level.
Now this is new and different, I should tell the listeners there's a concept called precision agriculture. It's been around for about 50 years and the ultimate logic of that is, I'm going to be precise down to the plant level.And so if you look at the industry, you'll see that there are now people building technology, irrigation systems that allow the grower to give treatment to individual plants.
If it's if this tells you anything, John Deere was on the cover of Wired magazine six months ago. When John Deere is on the cover of Wired magazine as one of the leading A.I. companies in the world, you know that farming has changed, and the CEO of John Deere recently said farming is moving from managing by the acre to managing by the plant.
We are on the climate podcast and so here is the moment that we'll link it back to what's the benefit for the planet of doing all of this? So give us a sense for how much water you can save or how much less fertilizers you can apply if you have this detailed knowledge on that sort of day by day, plant by plant basis.
Yeah, this is interesting. No one really knows. But once what's for sure is, it's going to happen. And the reason is there's no more arable land now. Different people debate this, experts way wiser than myself. But the consensus seems to be that all of the arable land that exists in the world, the sort of really fertile soil, the kind of Iowa's and Ukraine's, it's already being farmed.
And we're talking about a 40% increase in world population over the next 30 years. So you're talking about a massive increase in population, billions of people, more people than we have had in the last 2000 years. So you're talking about a massive increase in population and no more arable land. Now, to add to that, you will not be able to apply the levels of fresh water that are being applied.
Now, there's going to have to be some adjustments made. So you're not not going to be able to increase water consumption, you're going to actually have to decrease water consumption. You also likely have the situation where the fertilizers and pesticides that are common in farming right now, a lot of those are ultimately going to go away, at least not all of them, but a good deal of them.
And people disagree about how and when, but it's going to happen. So when you use the phrase, people say the phrase more with less. That is literally what is going to happen. You're going to have to do more with less. So I always say that sustainability in farming is not something that people do because they think it's inherently good. They don't have a choice. There's literally no choice but to do it.
Yeah, there is this convergence between what the planet needs and what the farmers want because of economic and scarcity of supply considerations. And we've seen that this year with the fertilizer scare and higher prices overall. And that's an interesting convergence. Before we talk more about your customers and why they are interested in your solution, I'd love to spend some time, for those who are listening, describing your solution.
So I've seen the demo videos and it's basically a tractor going through the vineyards, or a small car, I guess. I don't know what's the right way to call it, going through the vineyard with a camera, then linking to an AI system at the back end.
Well, it's, it's actually more, it's more compact than that. So we basically hitch a ride on anything that moves. That's the way we like to think of it. So what we are is a camera now. It's not a GoPro kind of camera. It's a camera that is built to accommodate the eye. And for those AI folks out there, they'll understand there's a euphemistically called the mind body problem, which is you cannot use an off the shelf camera with bespoke AI anymore.
They can use off the shelf AI if it exists with a bespoke camera. You have to build a camera and AI together. And so our camera was custom built to allow the AI to work. So it's a dual lens. It uses what's called a global shutter, and it has its own light source and you're getting kind of like each lens is getting five frames a second.
And what you're doing is taking images not unlike your cell phone images, RGV images, and then those are combined to create a three dimensional image of the plant. I'm going into this detail because I want the folks to understand that when I talk about imaging a plan, it's not the same thing that you normally think of when you think of imaging.
And so we image the plant, and the reason we do it to such a degree is that that allows the AI to analyze the plant at the pixel level that is captured on the camera. The camera is powered by whatever it's on. I mean, we've used tractors, ATVs. In fact, one of our investors is Kubota, one of the world's largest tractor companies in the world.
So they saw the premise of it behind it. And so we hitched a ride, capturing the data. Then that camera gets to a place where it can upload the data, and in the cloud, the AI then looks at the image. And of course, as I said previously, each plant is geo located. And the reason we do that is we will inspect that plant for as long as it's alive.
So we're going to be looking at that vine, for every vine that we look at - and we looked at about six, almost 7 million vines so far in three continents. It has a name and we're following that vine in the history and the performance of that vine. And then that's sent right back to the grower there. Dashboard shows them everything about that plant, to put it simply, so the viewers understand this in the fourth sense, we can tell a grower there's mold on that grape, on that cluster, on that vine.
So it's a level of detail at scale that doesn't exist anywhere in the world.
Is it actually real time or is there a latency between the moment you take the imaging and the moment you have the summary of the condition of the plant?
Yeah. It's within 24 hours right now.
There will be a day probably we're looking at 2024, perhaps as late as 2025 where it is real time, whereas the fact the data will be processed literally on the device itself and they should say to the viewer, the device is about the size of a small toaster. This is not a big machinery and it'll be processed on the device and it may in fact drive activity.
So you might say that it would inform me that maybe a sprayer or some other activities that necessitate a piece of an action by the grower while it's moving. So that's it. Yes, that's down the road.
Tell us about the AI part. So did you have to, you know, develop your own image recognition models or were you able to you know, there has been so much advancement in that space this year. Were you able to build off the back of the open source model or the openly available models that are out there?
Well, we were fortunate in that the company was spun out of Carnegie Mellon. And now there may be some folks from MIT and Stanford listening right now. But I'm going to say it, CMU was ranked number one in the world, artificial intelligence and robotics as well. Let's just say we're all in the peloton. We're all in the pack there. Right. And so it was developed over about a decade. Little fact about Carnegie Mellon is it's been in robotics for over 40 years and everyone understands the capabilities of the Robotics Institute, CMU, but it's early work at Carnegie Mellon. 40 years ago it was actually agriculture.
So it has a long history of that. And one of the things that the founders did is, you know, one of robotics claim to fame for CMU, among the things they've accomplished, is perception, is the idea of a robot knowing where it is at any given time and what it's seeing, so to speak. And the founders took the perception piece and carved that out, said, we're going to make that a service.
So the AI was developed at CMU, right? Neural net technology specifically for knowing, in effect, that it's looking at a plant, the features of the plant and so on. So we were fortunate to get a running start at this technology.
And in the training kind of the training data set that was used to I imagine you had to kind of at the beginning feed into its set of images and species to be able to, for example, map, you know, image signals to which classify into the right species right and perhaps into the certain different state of those of health states of those plants?
So that was done at Carnegie Mellon. And I give a shout out to Cornell, to the viticulture program, one of the leading viticulture programs in the world, shout out to Cornell that the original work was done when it was at CMU in collaboration with the Viticulture Program at Cornell, and they did great work together. So we had a data set from, you know, basically in upstate New York where we serve customers.
And that gave us a running start. And then it was our mission to get into a breadth of vineyards. In fact, again, part of the AI strategy is that you want volume and you want variety because you want robust models. So we specifically went after vineyards in the United States, in Europe and in South America.
And then we specifically also went after not just wine grapes, but table and even juice grapes. And now we've expanded into blueberries. So what you do when you know exactly.
That leads us to the next set of questions. What are the characteristic peaks of a vineyard that is most likely to be a good fit for your solution? Small, large, different types of grape, locations. I'm not sure what is the right way to cut the cake.
Yeah, that's. I will tell you, I'm not sure either. And because we have had success, you know, in a variety. Yeah, but I would tell you that what seems to be shaping up is two sets of customers that really get tremendous benefit from what we're doing but in different ways. One would be what I would call the high end vineyards.
These are vineyards that are probably smaller and they might sell a bottle of their wine for two or three or four or $5,000 a bottle. So I can't afford the products of our customers, but we're talking incredible wine, you know, in France and in California primarily. And then if you go to the other end, what I would call the industrial growers, these are growers of juice grapes, table grapes and wine grapes that are doing it to really tens of thousands of acres.
Now, they're both using the technology, but they're using it in slightly different ways. And probably the simplest way I could describe that is for the high end vineyards, they care about quality of yield and not necessarily the volume of the yield. So they will recognize the size of the yield so long as they can ensure that the yield meets their quality sort of profile.
And it's not just a it's not really a bar. It's more like a dot that you have to land on. And so in the case of the industrial growers, they're really looking at the size of the yield, you know, quality, but not to the extent that the high end vineyards, they want to make sure that they really understand their production process.
So the reason we're able to satisfy two very different customers is most vineyards just don't have this information at all. So for the time being, we're giving them a look inside what I call the black box, which is their farm.
How do you start and then progress conversations with customers? Do you go direct or do you work with distributors? How does that work?
Yeah, we're very fortunate now to have some partners. Most recently we closed a round of capital. Among the investors is a firm called Oeneo. It is a French company based in Bordeaux. Some of you, if you're vintners, probably have heard of Deum. It's a very popular cork for some of the world's leading vineyards, a company that was founded in 1836 and supplies 10,000 customers worldwide.
They saw us and said, we've been trying to provide similar services to our customers around the world and they do provide through one of their subsidiaries called Vivelys. And Vivelys is one of the leading providers of what I would call high end services to vineyards around the world. And they said, hey, this would be a great tool for us.
So through them we access customers. We have another partner in a company called Oppy. If in the United States, anybody that has shopped at Whole Foods has either purchased Sun View, has purchased Ocean Spray Table grapes, and they are to some extent supplied by Oppy. They are an investor and a partner and a customer and proud there. And then Kubota has about 4 million tractors used in any given day worldwide.
We also do our own approach to customers. We're fortunate, Nathan, in that a lot of our customers knocked on our door and it's still we haven't had to do as much direct selling as I've had to do in previous start ups.
What type of profile is doing sales at your company?
That's interesting. We're in the throes of expanding that team. So if anyone's listening who's interested, typically they're sort of a dual role. We call them director of growth and partnerships. Number one is there is some direct approach, but typically the direct approach is to very large vineyards.
And so you're talking about a B2B kind of sale where you're not just saying, hey, I want to service your hundred acres. Typically it might be to service your thousand acres or larger. So there is a direct selling component, but the bulk of the effort is around working through the partners to say, Hey, is there anybody in your portfolio that you think would be most likely to try this?
Because we're still at the early stage. This is, again, a new tech. And so we're able to tip through them, identify early adopters, and inevitably we're finding our partners say, oh, you want to go to X, Y, Z vineyard. They are always interested in trying new things. So that's been very successful to know.
Within the vineyards organization who is the person typically that will be in charge of assessing this type of new solutions?
It's funny, they usually fall into two categories. And for smaller vineyards, they're typically the same person. And so you have what I call the factory analogy. I spoke at an automation conference to a bunch of people in the factories, and I said, Think of the factory manager. So this is the person, the operations person who's responsible for the production of the crop for the year.
So their measured performance on yield quality and do they meet the specs to service their contracts. So that's really the operations manager and there's different titles and vineyards for that person. But think of the person who's really overall responsible for the practical aspects of getting the work done in the Vineyard. That's typically a first conversation. Then there's again, using a factory analogy, you might call the manufacturing engineer.
So this is the scientist, so if you will, who works with the operations team to ensure the quality of the grapes. So they are constantly testing, they're doing assessments during the year and they're typically highly qualified that are viticultural or viticulture lists typically have degrees of to Ph.D. we were for some very, very smart, talented people and they are agronomists.
They are, as they have been, in viticultural as horticulturalists even. They also have a say. It's funny for the latter group, I was always concerned that we would go in there and say to somebody, you know, a vineyard that's been around for 300 years and somebody with a Ph.D. who's been doing it there for 30 years say, hey, we're going to help you figure out how to help grow grapes.
And, you know, so you're always a little reluctant to say that but interestingly, it's really the experts who've really taken to this because now they have a volume and variety of detailed data on a continuous basis they never dreamed they would have. So it's been so far so good.
I'd love to ask you about your pricing model just to understand how it works?
It's a subscription. Let's just say that's very affordable. The simple way to describe it is, you know, when we look at how farmers, how growers buy things, they typically either buy equipment, finance that equipment, or they buy and purchase by the acre.
And we said, let's just forget all that and let's just charge it a flat fee. And we use what I call the cell phone model, which is you pay a flat fee and you can inspect as many crops as you want as often as you want. And with that flat fee, we give you a piece of hardware. You don't pay an extra fee for the hardware, just like a cell phone.
The price of that hardware, which isn't terribly expensive, is amortized into the contract. So the grower simply pays a monthly fee and they can put that device on anything that moves and they can take images of as many plants as they want as often as they want, as fast as they can. But it's and it's still a flat fee and that seems to have worked pretty well.
And it's part of our strategy as well, because we're an AI company and AI companies grow by data, and so if we metered data collection, they'd be inhibited. We didn't want that.
The one last question, you mentioned John Deere and some of the other sort of big precision farming technologies. Why does this technology emerge out of a startup like yours and not out of the R&D department of a John Deere or some of these other big companies?
Well, so, Nathan, I'll give you the answer that I teach in entrepreneurship. Years ago I went to Kodak and they have a museum. And in that museum, when Kodak was still in existence, there was a big piece of machinery that was built in 1972 and it was a digital camera. It was big about the size of a small refrigerator.
And you think Kodak, which went out of business some years ago, had 50% of the world's camera market. And you think why would a company that had 50% or more of the world's film market that invented the digital camera 25 years before they were introduced in the market, go out of business. And I think if anybody really wants a tutorial on this, read Clayton Christensen Innovator's Dilemma. Essentially companies exist to harvest the inventions of yesterday to maximize profit so they're optimized to extract value from what has been created previously. For them to move resources in something that's brand new, it changes the incentives. So big companies know this. I am not in any way casting aspersions on big companies.
Big companies have invested in us. We work with big companies. They understand that their structure may not be conducive to them seeing everything that's available. And that's why little startups will always exist because of folks like us, and other startups out there will always have opportunity because these big companies don't necessarily have the incentives and the structure to adapt to new, dramatically different new technologies.
I hope that that answer works.
For sure. I mean, who am I to judge? But yes, it resonates with me a lot. And it's a pattern that, you know, has been seen time and time again that incumbent companies find it hard to be as fast and nimble as startups, you know, that are coming out of the leading university departments in this specific sector.
I would give credit to Kubota. They have an innovation program and they recognize that. And they have innovation programs in Europe, the United States and Japan. And they're fully conscious of that fact, and make an effort to find innovations like us.
This one is the real last question. You raised a funding round recently. What key milestone are you trying to hit with that?
Sure. So we've moved out of what I would call the early stage. We've sort of graduated high school, if you will, as a startup. And now we're moving on in our life. And now we're at the stage where we have a product that works and it's commercial grade. And so our previous customers who were in the pilot stage are now using something that is a commercial grade product in that space, which I think lasts for a year or so.
It's what I would call the shakedown cruise, you know, where you take the brand new ship out to sea on its first voyage. So we're learning some things and acquiring some knowledge, both in terms of the capabilities of the product, but also being able to incorporate our customers' new ideas.
Yeah. So this money will allow us to really get through this phase of what I would call first learning with commercial use and then incorporate changes into the product and then allow us to come up with the design for yet another version that will likely be introduced in 2024 or 2025. We're excited. We've got wonderful customers, patient, thoughtful, incredibly smart customers.
So we're fortunate to be in that situation.
And you've mentioned that you are hiring in some key positions, so we'll put the link to your job openings page. And you know, hopefully some of our people listening or watching this will get inspired by your mission. Mark, it's been really fascinating to dive into this. We've hit our 30 minute mark. I hope it's okay with you. I hope to have you back here in a year's time and hear all about the progress that you've made. And we can maybe do this not around a cup of tea, but around a glass of wine.