Arduinos In The Grow Room: My Project

[odam2k]

I use the Arduino JSON library. Worth checking out if/when the time comes.

Looks like something I'll be interested in... Had a very quick look at the site, I think I might play with it tonight (later on) and see what it will take to replace my xml code.

Thanks, I think... lol

 

[bobinca]

I also use arduino json to return the configuration info from my server. It is very easy to parse the response of a network connection

bool httpGet(const char* url, JsonDocument* jsonResponse) {
  EthernetClient client;

  SERIAL_PRINT(F("  Connecting ...  "));
  client.setConnectionTimeout(4000);
  client.setTimeout(4000);
  if (!client.connect(serv, 80)) { 
    SERIAL_PRINTLN(F("  Connection failed"));
    client.stop();
    return false;
  }

  client.print(F("GET "));
  client.print(url);
  client.println(F(" HTTP/1.0"));
  client.print(F("Host: "));
  client.println(serv);
  client.println(F("Connection: close"));
  if (client.println() == 0) {
    SERIAL_PRINTLN(F("Failed to send request"));
    client.stop();  
    return false;
  }
  SERIAL_PRINTLN(F("OK"));
  // Check HTTP status
  char status[32] = {0};
  client.readBytesUntil('\r', status, sizeof(status));
  if (strcmp(status, "HTTP/1.1 200 OK") != 0) { 
    SERIAL_PRINTLN(url);
    SERIAL_PRINT(F("  Failed: status = "));
    SERIAL_PRINTLN(status);    
    client.stop();
    return false;
  }

  // Skip HTTP headers
  char endOfHeaders[] = "\r\n\r\n";
  if (!client.find(endOfHeaders)) {
    SERIAL_PRINT(F("  Invalid Response"));
    client.stop();
    return false;
  }
  if (jsonResponse) { 
    DeserializationError result = deserializeJson(*jsonResponse, client);
    if (result!=DeserializationError::Ok) {
      SERIAL_PRINT(F("  Failed to deserialize json: "));
      SERIAL_PRINTLN(result.c_str());
      client.stop();
      return false;
    }
  }
  client.stop();
  return true;
}

 

[odam2k]

Question about json, and in particular, this library...

With my XML parsing, in my main loop, I check the Serial port for available characters. If one is found, it is read, and checked for a number of things, including < and > to signify start and end of a TAG. This method processes the incoming data, one character at a time, and parses it from the XML into my variables.

Because I monitor the serial port continuously, I am able to receive unsolicited messages. For example, the Mega might need to write to the debug log table in the database, and it does so by sending the debug info to the ESP8266 via the serial port, simply sending XML. The ESP picks it up, parses the first tag, the table name, or Payload Type, and using that knowledge, picks off the rest of the data and writes it to the MySql Database.

At no time is the ESP8266 EXPECTING the debug info, it did not request it, therefore it is unsolicited...

From what I can see, at least with this library, is that I need to actively deserialize the data by making a specific call to do so, which implies I am waiting for that data...

Does this make sense?

Will this library allow me to receive and parse unsolicited data?

 

[odam2k]

There's that sound of crickets again...

Seems those crickets are visiting this thread...

There were 1,000 views in the last 19 days... That can't be ALL me

 

[odam2k]

I keep all my modules plugged into a USB port somewhere, so I can monitor the Serial output, which I use as a running log/debug window into the execution of the software. The sensor modules and autowater module are plugged into a laptop in the grow room, and I just remote desktop to that machine and I run 3 copies of the arduino ide, each set to different com ports, and I open the serial monitor for each, so I can monitor all three at once. My development machine has the Maintenance module, and the webserver module, as well as my test sensor module plugged into it, and using Visual Studio, I can open serial monitor for as many ports as I want...

So, all that is to say, I rely quite heavily on the serial output for debugging, and just for a general health checkup...

One of the widgets in Blynk is the Terminal. Just what it sounds like, a screen to view text sent from an arduino, and a one line entry area to send text back to the arduino. I've disabled the "send" input box, and am just using the receive box, and the AutoWater module is sending all my text, not only to the serial port, but also to the terminal in blynk! Now I don't even need to look at the serial ports any more...

I'm just loving this... every day brings new discoveries and ideas...

From this...

to this...

I'll need a way to tell it which module should be sending data, and when to start and stop. No sense wasting the bandwidth if I'm not even looking...

 

[odam2k]

It missed a few items, this is just a proof of concept, so no worries...

 

[odam2k]

Not sure how much work I'm gonna get done tonight....

 

[odam2k]

Well, it hasn't slowed me down... lol...

I'm thinking about the idea I mentioned where the AutoWater module would save the reservoir volume/depth to the sensor log table just like any other sensor....

The problem with that idea as it's stated, is the AutoWater module is really meant to do one thing, and one thing only, safely, and reliably water plants by turning pumps on and off. It is extremely important that it is both safe, and reliable, so no other tasks are allocated to this arduino. While it is watering a plant, it constantly monitors the reservoir depth, so to try to package that data up, pass it from the Mega to the ESP, then create the SQL, call the MySql connector to insert the record, blah blah... too much "extra" work when it needs to be paying attention to the pumps....

So, it can't happen every time it reads the ultrasonic sensor, and I really don't want to do it on a timed basis either.

My plan will be this... and I kinda like it...

Right now I have a sensor table which contains a record for every sensor I have, no matter what it is, thermistor, DHT22, Soil Moisture sensor...
I also have a module_port table, which is basically a list of all the available Analog Ports (1-16) on each module.

In order to know where the sensor is hooked up, and for a module to be able to find all the sensors hooked up to it, there is a "map" table, this links a sensor to a module/port

Example,

Modules
1 - Module 1
2 - Module 2

Sensors
1 - Soil Moisture
2 - DHT-22

Module/Ports
Module 1 / Port 1 / Analog Pin 0 / Power Pin D23

Map Sensor Module/Port
Sensor 1 / Module 1 / Port 1

So now when Module 1 scans the Map table, it knows that Sensor 1 is on Port 1, which it knows is Analog Pin 0, and the sensor is powered from Digital Pin 23


My plan is to add another field in the MAP table, i2cHostModuleId
By default this field will be NULL, but it it has an id, then the SensorModule, in this case, Module 1 will read the MAP tabse, and get the i2cHostModuleId (because the waterDepthSensor is an i2c sensor, the sensorId will be the i2cAddress, and Module 1 will simply contact the AutoWater Module using the Blynk Bridge requesting a reading. If the Autowater Module is busy doing something, it will simply ignore the request, otherwise, if it is free to do so, it will grab a reading and reply. This way I don;t have to write any code on the AutoWater Module to package up and save data to MySql.

 

[odam2k]

Got side tracked and started playing with a Switch Module, a Wemos Mega 2560 with a built in ESP8266, which will control Fans, Lights, etc.

I figured since there was some discussion of lights on other threads, I'd move this one up at least for a proof of concept... I'm pretty sure I've been overthinking it, as this solution seems very simple... either that, or this won't work?

So I have a relay hooked up to a digital pin on the Mega, and I'm using the "Normally Open" connection on the relay, so by default, the lights are off.

So if it's dark time, and something happens and the arduino reboots, the lights will stay off during the reboot... This is the desired behavior for flowering plants. For veg, it doesn't matter...

Two problems, and a solution for each...

Problem #1:
If the network connection is lost, we must continue to function.
Currently my Sensor Modules will go into an endless loop trying to connect to the network and load their configuration, and reboot after so many tries. That's perfectly fine for the Sensor Modules, since without the network, they can't save the sensor data to the MySql database anyways.

Solution:
We need to cache the module configuration as well as the configuration of any lights attached. If we reboot and cannot load the module configuration, or light configuration from MySql, then we will have to rely on the most recently cached version. This will actually work well, since the only reason it reads the configuration so often is to pick up on any changes, and if I were making changes, chances are, I'd notice that it was down.

Where do we cache this information?
SPIFFS is a good solution... Serial Peripheral Interface Flash File System, or SPIFFS for short. It's like having a disk drive to save files on.


Problem #2:
If the arduino reboots during lights on time, then the lights will shut off, and seconds later turn back on, something I'd rather not happen, it's called a hot start, generally the ballast should prevent hot starts.

Solution:
We need to cache the current time so that in the case of a reboot, we know how long we've been down, and can make sure we wait a specified period before we restart the lights.
Each light will have a config option for StartDelayTime and that way we won't need to rely on the ballast to prevent hot starts.
When the system boots up, if the lights are supposed to be ON right now, then it will check the SPIFFS for a file which is updated every minute and contains the current time. So when it reads this file during a bootup, it can compare the time in the file against the current time (It has an RTC module) and determine if enough time has passed to turn on the lights. Lets say you set the StartDelayTime to 5 minutes, and according to the timestamp in the file, only 3 minutes has passed, then we will delay a further 2 minutes before turning the lights on.
When we reboot, if the lights are supposed to be off, then no action needs to be taken at all...

 

[odam2k]

Because the ESP on the Wemos Mega board is only 1M memory, I was only able to create a 64K File System. That should be plenty of room.

I'm starting with the timestamp file, so I can start testing interruptions... so far it is switching within 60 seconds of the configured time, I just need to add the hotstart delay. Once that's working, then I'll add the configuration caches so the module can operate without a network connection...

 

[odam2k]

When I first looked at Blynk, I remember thinking how limited it was. It came with a fixed number of widgets, and there are no options for adding any third party ones.

I'm glad I didn't let that stop me, because I am blown away by how flexible those widgets are, and how thoughtfully they were chosen.

That's it... back to work...

 

[bobinca]

When I first looked at Blynk, I remember thinking how limited it was. It came with a fixed number of widgets, and there are no options for adding any third party ones.

I'm glad I didn't let that stop me, because I am blown away by how flexible those widgets are, and how thoughtfully they were chosen.

That's it... back to work...

I've been happy with Graphana and Blynk. I don't think anything is missing that I need.

 

[odam2k]

I'm making progress...

I have the SPIFFS set up on the ESP8266, and every minute, the Mega grabs the current time from the RTC, and sends a command to the ESP8266 via the serial port, using XML. The ESP8266 writes that timestamp to a file in the SPIFFS file system. It overwrites the existing timestamp each time.

When the ESP8266 boots up, in setup() it opens this file and reads the timestamp, and now it knows, within a minute, when the arduino was last functional.


Opening port
Port open
[M0] 2019/5/2 0218 > Grow by Wire
[M0] 2019/5/2 0218 > Free Memory: 1.16KB
[M0] 2019/5/2 0218 > Module Code Compiled on May 2 2019 at 0237
[M0] 2019/5/2 0221 > Server Code Compiled on May 2 2019 at 0216
[M0] 2019/5/2 0221 > MAC Address: CCE3:0C:7C:82
[M0] 2019/5/2 0221 > IP Address : 192.168.1.23
[M0] 2019/5/2 0221 > RSSI : -62dBm (76%)
[M0] 2019/5/2 0221 > Last Cached Timestamp: 2019/5/2 0212
[M3] 2019/5/2 0225 > Module has Sensors: 0, Switches: 0, and Lights: 1
[M3] 2019/5/2 0225 > Fetching Light Status from Server...
[M3] 2019/5/2 0226 > Module setup() Complete
[M3] 2019/5/2 0218 > Fetching Light Status from Server...
[M3] 2019/5/2 0219 > MH 400W on pin 8 should be OFF
[M3] 2019/5/2 0219 > Relay Pin LOW

We can calculate how long the arduino was down, within one minute. This will be used to determine if it is safe to turn the light on if it has a value configured for hotStartDelayMinutes

 

[odam2k]

My light in the Flower Tent comes on at 5am, except for today...

This is the second time in a couple weeks. The last time, I had set the ballast to "Super Lux", basically higher than the 400W setting. The next morning, the light did not come on, although I could her the ballast making a slight buzzing sound, so I knew it was powered... Apparently, some bulbs won't light when subject to startup with the higher power level. I switched it back to 400W level, and haven't had a problem...

Until this morning... I don't look into the tent very often, since my monitoring is telling me when a plant needs watering, so just maybe once a day for a general visual check. With the new light baffle over the air intake on the tent, I don't even see that light as a visual cue that the light is on or off...

So I'm working away (as usual) at the computer, on the Switch Module (which will switch the lights on an off), and noticed the light was supposed to be on, but it was not. I checked the timer, maybe it was off again, but nope, the ballast was making it's light buzzing noise, but no light in the tent. I unplugged the light ballast from power, and plugged it back in, same thing... Once more, unplugged, plugged it back in, and the light came on, whew!

So, new feature, the Switch Module will need the ability to remotely check a sensor on a Sensor Module in order to determine if the light is actually on. I already have photoresistors in each area...

Allowing a Sensor Module to receive "scan requests" will open the doors to some other cool features as well, but for now, it will solve this problem... The Sensor Module used to actually run the code for switches (and lights) but in streamlining tasks for these modules, I moved the Switch and Lights code to their own module. The Switches have the ability to be triggered by sensors (ie, fan on if temp > x) but now, the Switch module has no sensors on board. The ability to remotely request a reading will allow me to continue to use sensors to trigger switch changes, in fact, it opens the door to using ANY sensor on ANY Sensor Module.

So many ideas and changes to make, so little time...

 

[odam2k]

It works! This was configured with a 2 minute delay (120 seconds)

Opening port
Port open
[M0] 2019/5/2 14:showboating:14 > Grow by Wire
[M0] 2019/5/2 14:showboating:14 > Free Memory: 4.51KB
[M0] 2019/5/2 14:showboating:14 > Module Code Compiled on May 2 2019 at 1448
[M0] 2019/5/2 14:showboating:17 > Server Code Compiled on May 2 2019 at 14:07:41
[M0] 2019/5/2 14:showboating:17 > MAC Address: CCE3:0C:7C:82
[M0] 2019/5/2 14:showboating:17 > IP Address : 192.168.1.23
[M0] 2019/5/2 14:showboating:17 > RSSI : -70dBm (60%)
[M3] 2019/5/2 14:showboating:20 > Module has Switches: 0, and Lights: 1
[M3] 2019/5/2 14:showboating:20 > Fetching Light Status from Server...
[M3] 2019/5/2 14:showboating:26 > MH 400W on pin 8 should be ON
[M3] 2019/5/2 14:showboating:26 > Delay for 106 more seconds to prevent Hot Start
[M3] 2019/5/2 14:showboating:26 > Module setup() Complete
[M3] 2019/5/2 1414 > Fetching Light Status from Server...
[M3] 2019/5/2 1415 > MH 400W on pin 8 should be ON
[M3] 2019/5/2 1415 > Delay for 56 more seconds to prevent Hot Start
[M3] 2019/5/2 1414 > Fetching Light Status from Server...
[M3] 2019/5/2 1415 > MH 400W on pin 8 should be ON
[M3] 2019/5/2 1415 > Relay Pin HIGH


The next task is going to be a little bit harder, I need to modify the code so it does NOT require a network connection to work. Obviously it needs at least ONE connection at some point so it can load its configuration from the MySql database, but once it does, it will cache the configuration info to the SPIFFS file system EVERY TIME the configuration is loaded successfully.

Instead of waiting in infinite loops and rebooting like the Sensor Modules, which this code is based on, it needs to immediately load the cached configuration info when it cannot do so over the network, and continue on as normal. Everything must be self contained on this module so it can function correctly without outside influence...

I'm pretty close now...

 

[odam2k]

I'm still waiting for my "test plant" to get thirsty so I can watch the Automatic Watering module do it's thing...

Seems my Veg plants don't dry out nearly as quick as my Flowering ones, makes sense... I didn't want to test this on a flowering plant, cause it might water it in the dark, and I wouldn't be able to check on it.

It's gonna be a while yet, a couple days I think... It's plant #73, right in the middle...

 

[bobinca]

I like how you keep track of the age of the plants.

As for config, I do pretty much the same thing. My config is stored in EEPROM as json and returned from the network as json

1) set defaults from code
2) load config from EEPROM if it exists
3) get config from network (try up to 10 times with 30 second delay, the first try after reboot seems to have a 10% fail rate)
4) if I get a config from network then save to EEPROM (only write bytes that have changed, so if the config didn't change EEPROM will be saved from a write).
5) check for new config once per hour from network

 

[odam2k]

Storing the "cutting date", "veg date", "flower date" and "harvest date" for each plant...

lets me visualize it like this...

 

[odam2k]

As for config, I do pretty much the same thing. My config is stored in EEPROM as json and returned from the network as json

I presume you are aware EEPROM has a limited number of writes available, and you've calculated that it isn't going to be a problem?

This is why I love the Wemos Mega's, they are a full Mega 2560 PLUS they have an on board ESP8266, all for less than $15 from banggood. The Mega does all the work, and the ESP is like a network adapter/server. The ESP can also run the SPIFFS file system, web servers, MySql connector, etc...

 

[odam2k]

The next task is going to be a little bit harder, I need to modify the code so it does NOT require a network connection to work. Obviously it needs at least ONE connection at some point so it can load its configuration from the MySql database, but once it does, it will cache the configuration info to the SPIFFS file system EVERY TIME the configuration is loaded successfully.

Instead of waiting in infinite loops and rebooting like the Sensor Modules, which this code is based on, it needs to immediately load the cached configuration info when it cannot do so over the network, and continue on as normal. Everything must be self contained on this module so it can function correctly without outside influence...

I'm pretty close now...

Closer than I thought...

When I read the config from the database, for example, the module configuration, the ESP8266 reads the data from MySql and writes it to the Serial port for the Mega 2560 to read. This is done by packaging it into XML with something like this:

open cacheFile on SPIFFS for write (overwrite)

If (connectToSql)
{
    Serial.println("<MODULECFG>")
    write header <MODULECFG> to cache file
    Execute SQL query eg. SELECT * FROM tbl_module where moduleId=_moduleId
    For each row returned
    {
          Serial.println("<fieldname>fieldvalue</fieldname>")
          cacheFile.println("<fieldname>fieldvalue</fieldname>")
    }
    Serial.println("<MODULECFG>")
    write footer </MODULECFG> to cache file
    close cacheFile on SPIFFS
}
else
{
    open cacheFile on SPIFFS for READ
    Loop through file read each line and send to serial for Mega 2560 to read
    Close cacheFile on SPIFFS
}

So if it can, it fetches the configuration info from the MySQL database, and if not, it just spits out the previously saved configuration... Very simple solution, very little extra coding....