Arduinos In The Grow Room: My Project

[DrewT]

Here's my initial thoughts. Understand my background is in operations development and performance testing so forgive me if anything sounds too critical .

• PG2, PP4: Is this specification purposely excluding wired communication between servers and modules? If so, consider creating a non-goals section and explicitly specifying.
  
  
• PG5, PP6: Are you planning to add versioning information to your firmware and schema? If not, how will you keep track of which firmware revisions work with active schema
  
  
• PG5, PP9: How are you planning to deal with data consistency during network outages or database upgrades? When can you expect the sensor data not be logged to the database?
  
  
• PG6, PP3: Does this mean each column of plants can only associate with one sensor? Does this design account for your garden if you chose to orient plots in circle or line (opposed to a grid)? How can you ensure correct association of sensor data in non-uniform plot layouts?
  
  
• PG7, PP1: Are you saying all sensor modules will have 16 ports? Are you limiting MCU selection to Atmel Megas?
  
  
• PG8, PP1: Module is redefined here after citing it in the terms section above. Unless a module can be more than one thing, should these sections be combined?
  
  
• PG8, PP2: Is a server a module then? The hardware is a single physical unit so I may be confused if I didn’t know this ahead of time.
  
  
• PG10, PP2: Is static IP assignment required? Perhaps state that explicitly if it’s the case.
  
  
• PG10, PP3: Is there no option to store module config locally? If a network connection to the DB fails how will the module behave?
  
  
• PG11, PP2: Are you restricting the web server to an ESP-01? This has 1K of SRAM and is the least capable 8266 package commonly available. Do you have a scalability story for DB growth?
  
  
• PG12, PP2 : I’m confused, is each ESP-01 hosting its own database? PG18 seems to confirm this. Are logins specific to each ESP or is auth happening at the DB as well?
  
  
• PG14, PP2 : Duplicate field name ‘dateAdded’
  
  
• PG18, PP4 : DB maintenance is performed by a remote process correct? If so, how do you plan to orchestrate deployment and versioning control? What happens when network failure occurs or the maintenance server is taken offline?
  
  
• PG20, PP1 : Will sensor readings be transmitted over the wire as whole numbers (sensorRawValue) and converted to other types (sensorCalcValue) on the web server?
  

Open questions:

• How do you configure modules to connect/report sensor data?
• What happens when a sensor module dies and you need to replace it with the same config?
• What happens when the server runs out of SRAM/Flash to store DB info?
• What is the maximum range you expect to have between modules and servers?
• What other dependencies are there in terms of power and connectivity for each component type?

Again, great start and thanks for sharing!

 

[odam2k]

Here's my initial thoughts. Understand my background is in operations development and performance testing so forgive me if anything sounds too critical .

• PG2, PP4: Is this specification purposely excluding wired communication between servers and modules? If so, consider creating a non-goals section and explicitly specifying.

Keep in mind I am not building for scalability here, as I know the scale of my grow, and that it won't get much bigger, although so far my design could accommodate double without issues.

• PG5, PP6: Are you planning to add versioning information to your firmware and schema? If not, how will you keep track of which firmware revisions work with active schema

Yes, once I am finished with my current pace I will. As it is, because I'm developing on a live system, I have to keep the latest versions everywhere... Once I slow down, then simple version numbering system with database checks. It's not likely I'll ever end up with it in a mixed state, so this is of low importance, hadn't put any thought into it yet I've done similar checks in a number of projects in the past.

• PG5, PP9: How are you planning to deal with data consistency during network outages or database upgrades? When can you expect the sensor data not be logged to the database?
  

Sensor Data is non critical. It gets logged every 30 minutes, and if it misses one, two, even three, it's not the end of the world. There is no store and forward here, although, when the Module (2560) reads the sensor value, sends it via serial to the Server (esp8266), which in turn creates the proper sql statement and executes it on the MySql database server running on a laptop. Then the server queries the database to get the record id of the entry it just made, and if not successful, will try saving the data 2 more times (3 in total). If Successful, it reports back to the Module (2560) with the log record id, if it failed, it reports that as well. So the Module knows whether or not the data was saved. It just doesn't really care

• PG6, PP3: Does this mean each column of plants can only associate with one sensor? Does this design account for your garden if you chose to orient plots in circle or line (opposed to a grid)? How can you ensure correct association of sensor data in non-uniform plot layouts?

Correct, right now only one sensor per plot. While I've always been aware of this limit I never thought of it as a problem, I don't have a scenario in mind that cannot be accommodated now. However, this does seem like something I should fix.
Again, I'd never given it any thought...

As for the grid, each plot could be physically located anywhere, in fact, the web page displaying the layout is flexible enough that with a tiny bit of html editing, you could put any plot anywhere you like... Again, the beauty of specifically NOT designing for other peoples use, I don't have to accomodate scenarios that I'm not likely to see.

• PG7, PP1: Are you saying all sensor modules will have 16 ports? Are you limiting MCU selection to Atmel Megas?

I suppose I am saying that. Right now they do. Once again, my lack of scalability ideas are due to true lack of scalability I have no need...

• PG8, PP1: Module is redefined here after citing it in the terms section above. Unless a module can be more than one thing, should these sections be combined?
  

Poor writing on my part. The first one is supposed to simply define a module, what it is, in simple terms, just a box... The second was to define it hardware wise, the 2560, and esp8266, how they connect, etc... will clean that up...

• PG8, PP2: Is a server a module then? The hardware is a single physical unit so I may be confused if I didn’t know this ahead of time.

Yes, every mcu is a module. A Mega/ESP board is a Module, more specifically, a Sensor Module. This Module has an attached Server, the ESP8266 There is no basis for this naming convention other than that's what I called em. A standalone ESP Server would be considered a Server Module, but they are all Modules.

• PG10, PP2: Is static IP assignment required? Perhaps state that explicitly if it’s the case.

Not required, but in my case I find mDNS extremely unreliable (probably something on my network, haven't looked into it) so rely on IP addresses to access the web servers on them (for OTA Updates, remote Resets, forcing sensor rescans, etc)

• PG10, PP3: Is there no option to store module config locally? If a network connection to the DB fails how will the module behave?

Aha, yes, if a module boots up and cannot connect to WiFi or cannot connect to MySql to load it's configuration, then it will continue to retry. If it fails a preset number of times (Id have to go look) then it reboots the entire module and starts the process over.

If it cannot load it's configuration, then it couldn't save any readings, so it doesn't matter... The little RGB Led on the front flashes like a police car when something like this happens..

• PG11, PP2: Are you restricting the web server to an ESP-01? This has 1K of SRAM and is the least capable 8266 package commonly available. Do you have a scalability story for DB growth?

Not restricting, but not investigating upgrades either, since so far, I'm well within the limits of the hardware.

• PG12, PP2 : I’m confused, is each ESP-01 hosting its own database? PG18 seems to confirm this. Are logins specific to each ESP or is auth happening at the DB as well?

No, each ESP-01 has the Arduino MySQL Connector library loaded, and can talk directly to the MySql Database Server running on the laptop.

• PG14, PP2 : Duplicate field name ‘dateAdded'

Thanks

• PG18, PP4 : DB maintenance is performed by a remote process correct? If so, how do you plan to orchestrate deployment and versioning control? What happens when network failure occurs or the maintenance server is taken offline?

Again, not critical. The only issue would be that it does not water a plant that should be watered. There should be lots of leeway here.

• PG20, PP1 : Will sensor readings be transmitted over the wire as whole numbers (sensorRawValue) and converted to other types (sensorCalcValue) on the web server?

Sensor reading to sensorRawValue (int) is done on the Module (2560) which then does the conversion to sensorCalcValue (float) and passes both of these values, along with lots of other data, to the Server (8266) which does some data lookups, then creates the SQL required to insert the record into the log table.

Open questions:

• How do you configure modules to connect/report sensor data?

Each sensor is connected to a Module via a Port. That connection needs to be recorded in the database, so that when a module asks for a list of attached sensors, it gets a list of JUST the ones attached to it, and which Port it is on.

• What happens when a sensor module dies and you need to replace it with the same config?

If it's a direct replacement of a bad sensor, I would simply replace the sensor, and number it as the old one. If it is just another Sensor of the same type, then I just need to change the ID in the database. The module needs to know specifically which sensor it is, because of calibration and sensitivity...

• What happens when the server runs out of SRAM/Flash to store DB info?

Nothing is stored on any of the mcu's except a SensorId (i2c address) in the EEPROM for I2C Sensors. Everything is written to the database in real time.

• What is the maximum range you expect to have between modules and servers?

Well, I'm in an apartment building, 8th floor, and don't have far to go, but there's a lot of RF noise, so even from the next room I had to put 2.4Ghz antennas on the Mega 2560's, and I also monitor the RSSI of the network signal, and so far no problems....

• What other dependencies are there in terms of power and connectivity for each component type?

Each Sensor Module is powered via the external power plug with either 9v or 12v. The ESP modules are currently powered via USB but that's because they are on my desk, actively being worked on

 

[odam2k]

Whew! That was awesome, thanks for your input...

I tend not to think of scalability in THIS particular project since I have no plans to ever take it any further than my own garden. I've been a Senior Software Engineer for many years (nearly 40 years programming), even leaning toward architecture the last few years, and worked on some fairly substantial projects, so I am familiar designing for scalability, and more importantly, for users...

Now that I am on disability, and doing this for fun, not profit, I can safely ignore anything outside my current scope, with the safe knowledge, that should I ever need to upgrade the system in the future, I personally know the programmer lol

I know that if I were trying to build an open source system, I'd be so bogged down in "what ifs" that I wouldn't have anything working at all


So on that note, you have some great suggestions which I will follow up on, and some which perhaps help you think of your own project...

Thanks again

 

[odam2k]

Your question was:

• What happens when a sensor module dies and you need to replace it with the same config?

My Answer was...

If it's a direct replacement of a bad sensor, I would simply replace the sensor, and number it as the old one. If it is just another Sensor of the same type, then I just need to change the ID in the database. The module needs to know specifically which sensor it is, because of calibration and sensitivity...

To actually answer your question:

That's the beauty of my "Sensor Wiring Harness Shield", just unplug the Mega board from the Shield, and that's the only connection you need to worry about other than usb and power. It's just a DIY Perfboard with all the pins through the board and broken out on the top to provide Signal/Power/Ground for all the sensors, I2C, LED, LCD, RTC, etc...
Replace the Mega with a new one (I keep 3 spares) update the MAC address in the database so it can load the correct Module Configuration. Also, update the MAC on the router for the static IP address.

 

[drummer69]

Im off to work and will def read it thoroughly x2 whenI get home Odam2K

 

[odam2k]

• PG6, PP3: Does this mean each column of plants can only associate with one sensor? Does this design account for your garden if you chose to orient plots in circle or line (opposed to a grid)? How can you ensure correct association of sensor data in non-uniform plot layouts?

Sorry, I missed your wording....
No, a column is not limited, but a Plot is. There can be multiple Plots in a column, it's just an arrangement...

But yes, a Plot can only have one sensor at present...

The web page "Plot Map" which shows the overview of the grow room is going to be the biggest problem here, since it has to display the sensors data in each square....

So this will need to be modified to be able to show 2 sensors rather than one...

As for configuring two sensors to be on the same plot, no changes are required, it's done using a map table like so:

You can see Sensor #46 is located in Plot #22
If I add another entry, say 47,22 then Plot 22 would have 2 sensors, but that's the easy part...

Need to make sure that multiple sensors are included when requesting attached sensor list.
Need to make sure you can mix and match different sensors, ie, analog/digital/i2c, or multiples of the same...
Need to store results in Log, I suppose each Sensor has it's own log entry.
Then the big job, displaying the results of multiple sensors on a single plot.

If 2, why not 3? 4? Perhaps best if there is no real limit other than practicality...

I see a redesign of the Plot Map screen coming up... just not sure what will change....

 

[odam2k]

Let's take this right back to the Plot...

Plot
A Plot defines a place, usually a physical location, sometimes a virtual location. Each Location is divided into Plots, arranged in Rows and Columns.

For example, my Aqua Cloner has 4 rows of 6 holes for cuttings, so 24 Plots.
My Veg Tent has 3 rows of 4 “spaces”, or 12 Plots
My Flower Room has 3 rows of 6 “spaces”, or 18 Plots.

Those are physical Plots, each one defined by the Location, Row, and Column.

Rows start at 1, and Row 0 is reserved for Virtual Plots. These are simply placeholders for Sensors which don’t measure plants themselves, such as Temperature, Humidity, and Light Levels. These Virtual Plots will be used to determine where to display these readings on the Web Pages. Each location has one Virtual Plot for each column.

If you wanted to accomodate a circular pattern for your pots, you could still do so by envisioning a 10x10 grid (rows and columns) and just picking the plots where a pot would sit in that circular pattern...

So, the idea is that a Location should have a variable number of Rows and Columns (easy enough so far)
Plots currently are numbered sequentially from 1 to however many plots theree are, so this does not leave room to change the number of rows or columns of the Veg Tent, as this would change the Plot ID's in the Flower Room...

Veg = 3 x 4 (Plot ID 1-16 incl row 0)
Flower = 3 x 6 (Plot ID 17-40 incl row 0)

If we add a row in Veg, then the Plot ID's need to change in the flower room... not good...

I suppose if the actual Plot ID number was never presented to the user, then it wouldn't matter if it was sequential or not., Just show the Plot Description in any user interface. Now you can add plots without regard for the order you enter them, and they can still be sorted by location/row/column for the UI.

Now I can start on the Plot Map web page, first, finding the best way to display the plots, then I can worry about how to display multiple sensors in a plot. I think I'll increase the size of the Plot box in the page though to accomodate more data.

 

[DrewT]

and some which perhaps help you think of your own project...

Indeed! Admittedly I'm taking the open source route and thinking like a solutions provider. The benefit is a wider audience and impact, the drawback is much added complexity as you correctly identified. I really dig your applied experimentation approach and I acknowledge the value it provides folks like me that need to get their butts out of the abstract! Trust me, I'll bust my specifications out soon and hopefully join you at the table as a contributor! I say this with all respect and gratitude, I hope to convince you with action that greater collaboration is in our mutual best interest .

If 2, why not 3? 4? Perhaps best if there is no real limit other than practicality...

Just a thought, any chance you could associate one or more virtual plots with a concrete plot? Then perhaps you could associate a sensor (virtual plot) with one or more pots (concrete plot) in a way that isn't bound by a physical placement association.

 

[odam2k]

Just a thought, any chance you could associate one or more virtual plots with a concrete plot? Then perhaps you could associate a sensor (virtual plot) with one or more pots (concrete plot) in a way that isn't bound by a physical placement association.

The only place I have a problem is displaying it on the web page, so I don't think that will help....

The nice thing now is I can, at one glance, see ALL my sensors and whether they are within target range... It fits on one screen, and auto refreshes... Don't want to lose that...

 

[odam2k]

Perhaps a Sensor has a self contained UI object, so it doesn't matter where it is displayed, it will always look the same. So rather than displaying a page of Plots, I will display a page of Sensors, arranged by plot... (Location/Row/Col)

Hmm....

 

[DrewT]

Templatized sensor presentation framework, I like! I hope I'm not taking away from your work. I have a habbit of over-engineer stuff so don't let the complexity get in the way of enjoyment or an applied solution!

 

[odam2k]

Nah, I'm just thinking out loud too... It's a more difficult decision because I'm creating a TON of work, it just isn't solving any real world issues for me. If I'm trying to solve a problem, I have no fear when it comes to ripping code apart and spending days fixing it, but right now, I've got more productive things I could be doing...

I'll spend a little bit more time thinking about this, I like the idea of the presentation end being sensor driven rather than location driven.

Hmmm....

 

[odam2k]

Ok, enough time on that... Too much work, not enough return...

 

[odam2k]

It works! I just got notified that I have a plant that needs watering... I've adjusted the amount of water so that it should need water every four days or so. Now that I'm more comfortable with the notification, perhaps tomorrow I'll install a water hose on that plant and tell the system it can water the plant on its own. I look forward to the first time it automatically waters a plant!

 

[drummer69]

That is a major step in the proper direction my friend
I've already created a complicated monster on paper don't know how feasible the software side of it will be but the hardware side and such maybe 12-16 hrs of work

 

[odam2k]

Of all the Grow Room Automation Tasks, the most obvious one is one I have not tackled yet. The Grow Lights. Simple, plug them into a relay, and switch them on and off every 12 hours...

Well, I guess the idea that they are 110v and 400W each, and I want to switch it with that tiny little Arduino, made me nervous. I don't know why, there's nothing overly dangerous about it.

So, let's give it a go...

What are my concerns?

Well, timing I guess... With one of those timers you plug into the socket, they are very reliable, and only have issues if the power goes out, and even then, they still work, just delayed by the time the power was out... With an Arduino controlling the lights, it would be based on time of day, not elapsed time. I'd set a Lights On time, and a Lights Off time, and the Arduino, using a RTC module, will always know what time it is, and whether the lights should be on or off.. Sounds simple enough....

Remember, a relay is a switch, and I already have code for switches, however, I need this switch to be triggered by a clock, not a sensor.

So I need to add a new type of sensor, a clock

This "sensor" can drive the relay based on limits, but rather than triggering at say 75%, it will trigger at a time, say 11pm.

Right now I have a "switch" that controls the fan on the humidity dome over my cloner. That "switch" is driven by a humidity sensor, so when the humidity goes above 95%, it turns on the fan, and when it falls below 90%, it turns off the fan.
Now, I need the "switch" which controls the Grow Lights to be driven by a "clock" sensor, so when the time is between 5pm and 5am, the lights go off, and between 5am and 5pm, the lights go on.

There, designed the process in the time it took to write this post

 

[odam2k]

That is a major step in the proper direction my friend
I've already created a complicated monster on paper don't know how feasible the software side of it will be but the hardware side and such maybe 12-16 hrs of work

Everything in the paper is already working on the system. I don't plan ahead much, so trying to put it to paper before actually implementing it doesn't make sense especially since I'm not following standard development practices during this project. I'm in super rapid development mode I don't even know what I'll be working on an hour from now....

 

[DrewT]

Well, timing I guess... With one of those timers you plug into the socket, they are very reliable, and only have issues if the power goes out, and even then, they still work, just delayed by the time the power was out... With an Arduino controlling the lights, it would be based on time of day, not elapsed time. I'd set a Lights On time, and a Lights Off time, and the Arduino, using a RTC module, will always know what time it is, and whether the lights should be on or off.. Sounds simple enough....

Just a reminder... I already have all of this done. DS3231 over i2c, event handler chain, rudimentary scheduled task structures, non-vol storage into EEPROM for state recovery. Let me know it you want me to link you directly to the code

 

[odam2k]

I'm good thanks... Already created a new "virtual sensor" called "Clock" with three "values", Hours since midnight, Minutes since midnight, and seconds since midnight...

My switch can be triggered by any of those 3 values... Most of the code is already in place, just needs some testing and tweaking...

 

[odam2k]

Backing up....

I believe lights are important enough to command their own subsystem. Rather than being generic switches, if I treat them specifically as lights, then I can have better control over them. For example, I can use a photoresistor along with the lights to make sure that during lights out, it is dark, you didn't accidentally leave another light on etc... more later...