The first week of PFD lesson involves the objectives of undertaking real-life IT projects by making use of
scrum (Lightweight Framework) via different challenge statements. My team was assigned the challenge
statement
of "Data Collection Applications".
After a brainstorming session, we
arrived at a concept that involves utilising both a mobile phone and a wristband to monitor the wearer's
location. Additionally, we also plan to leverage the built-in Bluetooth functionality to communicate with
other nearby devices. However, this idea concept is quite similar to an existing application called
SGTRACETOGETHER. As a
result, my team is looking forward to discuss with our lecturer to refine and solidify it.
After meeting with our lecturer, our team decided to discard our prior concept because it did not
accurately correspond with the challenge statement. Subsequently, we have opted to redirect our efforts
towards the development of a mobile Arduino
pathway tracker that uses GPS. The device comprises an ESP32 MCU Module, which has Wi-Fi and Bluetooth
capabilities, as well as a GPS tracker to follow the user's movements.
Additionally, there will be
an AWS
API Gateway connection between the ESP32 and AWS DynamoDB, which will be used to store the location data
gathered by the GPS. Finally, to plot the course that the user has travelled, the Python library Matplotlib
will be utilized. While the current sprints are focused on research, this important work is essential to
ensure that my team and I have a solid foundation for development. Once we have obtained the required
materials, I will be able to transition to development sprints and start building our project.
SEER's Product Box
This graph below outlines the resources required in Project SEER
- How the product works -
Since the ESP32 module has a built-in Wi-Fi component, it can transmit the location tracks captured by the
GPS to the AWS API Gateway. The AWS API Gateway then routes the data to AWS DynamoDB for storing all the
captured locations. To monitor the security guard's patrol path, the staff can access the AWS DynamoDB via
the AWS API Gateway. Upon obtaining the data, they can use the Matplotlib library to plot a pathway graph
map on
Python. Matplotlib is a Python library that provides a comprehensive set of tools for creating data
visualizations.
- IBM Enterprise Design Thinking -
Apart from that, after completing the IBM Enterprise Design Thinking course, I learned how to apply the five
stages of the design thinking process to solve real-world problems: empathize, define, ideate, prototype,
and test.
I also learned how to collaborate with others and use feedback to iterate on my ideas.
Design thinking is a powerful tool that can be used to create innovative solutions to a wide range of
challenges.
Engage in Empathetic Understanding: I learned the art of empathizing with end-users, delving deep
into
their
experiences, needs, and emotions, allowing the benefit to better comprehend their perspectives and pain
points.
Precisely Define Problem Statements: A crucial skill & ability to precisely define problem
statements, ensuring that the challenges at hand are thoroughly understood and accurately framed.
Cultivate Creative Ideation: The course encouraged creative ideation techniques and the
importance of
generating a
plethora of innovative ideas & concepts to tackle the defined problems.
Master Rapid Prototyping: Being proficient in creating quick, user-focused, and iterative
prototypes to validate and refine ideas and concepts efficiently. It is a vital skill for design
thinkers as it helps them move from abstract concepts to tangible, testable representations of their
solutions, ultimately leading to more effective and user-centric designs.
Methodically Test and Iterate: The course taught me to methodically test prototypes, gather user
feedback,
and iterate on my solutions, ensuring they evolve and improve continually based on real-world input.
Finally, after a week of waiting, I was finally able to get my hands on the ESP32 Nodemcu, a breadboard, and
a couple of jumper wires. This mark the start and beginning of my journey to create the SEER GPS Tracker.
Despite not having the GPS Tracker device yet, i have initiated the process of conducting and testing the
evaluation of the ESP32 Nodemcu's functionality to ensure that it is in a proper working condition.
- Arduino Board Manager -
- Blink Start Code -
To begin testing the functionality of the ESP32 Nodemcu development board, I utilized the Arduino IDE,
an open-source integrated development environment that enables users to write, compile, and upload code to
Arduino boards. Since the ESP32 Nodemcu board is compatible with the Arduino IDE, I decided to use this
application to test its functionality.
Secondly, to ensure that my ESP32 Nodemcu board is properly supported by the Arduino IDE, I installed both
the Arduino
and Espressif Systems board packages for the ESP32. These packages contain all of the necessary files for
the Arduino IDE to compile and upload code to ESP32 boards. This is important because different Arduino
boards use different microcontrollers, and the Arduino IDE needs to know which microcontroller it is
generating code for in order to produce the correct machine code.
Lastly, i made use of an online simple testing blink start.ino file for testing. Since the ESP32 Nodemcu
has a built-in LED that can be controlled using the same digital pin as the Arduino board. To test the
functionality of the ESP32 Nodemcu, i uploaded the blink start code to the ESP32 Nodemcu and observe the
behavior of the built-in LED. If the LED blinks on and off every second, then the ESP32 Nodemcu is
functioning properly.
Moving onto week 4, although we were unable to obtain a GPS tracker, my team and I continued to work on
finding solutions for plotting captured pathway locations on a map. We discovered a website guide: a GPS
tracker that uses an SD card to record GPS locations, which can then be
used to plot the coordinates on a map using a Python script with the Cartopy library. Since the website is
relevant to our GPS location project, we decided to use the website as a fundamental foundation to guide,
assist and enhance our SEER Project.
At the current moment, my team and i are limited to only ESP32 Nodemcu and the python script guide (from
makersportal) that is available to
be worked and improved on. Hence, I decided to fill in randomly generated latitude & longtitude around Ngee
Ann Polytechnic in a csv file to try and test if the python script is capable of plotting
the Geo locations on a map.
To plot the pathway map, I used Jupyter Notebook to compile and execute the Python script. Initially, the
execution failed because the Cartopy library was missing. (Cartopy is a Python library for geospatial data
processing and visualization. It is built on top of the Matplotlib library and provides a variety of
features with geospatial data and support map projections.) Thus, using the anaconda prompt, i installed the
library via pip install cartopy.
- Python script guide from Makersportal -
- How the python script works -
It works by parsing the GPS coordinates from a CSV file, and then formatting them for use with
Cartopy. Next, it creates a Cartopy GeoAxes object and projects it using the coordinate reference system
(CRS) of a street map. It then sets the extent of the map, adds the street map image, and plots the GPS
points.
The first day of the hackathon, the long-awaited GPS Tracker finally arrived. I quickly got to work and connected it to my ESP32 Nodemcu to set the integration of the C++ code to bring the entire GPS Tracker system up. However, upon executing my first code sketch, the GPS Tracker failed to function.
The root cause of the malfunctioning GPS Tracker came from an error in referencing the wrong version of the ESP32 Nodemcu pin layout. Thus, leading to an incorrect pin configuration for the GPS Tracker. This mismatch between the expected and actual pin connections prevented the ESP32 Nodemcu from effectively establishing a connection and communicating with the GPS Tracker, rendering the entire system inoperable.
After some alterations, the output initially showed "unknown" location but displayed the correct date. However, rerunning the code caused a glitch, and the date reverted to 2001. Suspecting that there might be a CMOS battery issue, i researched more about the GPS device. The online research revealed that the Neo-M6 GPS HAT with Antenna for Raspberry Pi lacks a CMOS battery for date maintenance. It relies on the Raspberry Pi's power supply, losing date and time information when the Raspberry Pi is turned off.
- ESP32 Nodemcu C++ Code -
- Day 2 -
Onto day 2 of the hackathon, due to time constraints in getting the Raspberry Pi to work, our team focused on plotting a map using generated GPS coordinates of latitude and longitude.
To visualize the pathway traversed using the captured GPS coordinates of longitude and latitude, a Python script is employed, leveraging the functionalities of both Matplotlib and Cartopy libraries. This combination of tools enables the creation of an informative and visually appealing map that effectively conveys the movement patterns captured by the GPS data. Thus, allowing an evaluation to monitor potential areas of improvement or certain blind spots / unpatrolled sections of the security guard's route.
What is Matplotlib?
Matplotlib serves as the core framework for generating the map, providing a versatile platform for constructing and customizing various plot elements. Its extensive repertoire of functions and capabilities allows for precise control over the map's appearance, including the selection of projection, scale, and styling of geographical features.
What is Cartopy?
Cartopy, on the other hand, brings specialized expertise in handling geospatial data, seamlessly integrating with Matplotlib to enhance the map's geographical accuracy and context. Its advanced features enable the incorporation of coastline data, country borders, and other relevant geographical information, transforming the map into a comprehensive representation of the real world.
- Python Code for pathway visualization -
- Data in CSV file -
- Output from Python Code -
- Day 3 -
Moving to the 3rd day of the hackathon, our team will present our GPS prototype, a solution designed to revolutionize security guard patrol monitoring in the security industry. Our innovative approach effectively addresses the challenge of recording security guard prowl paths, ensuring comprehensive coverage and mitigating potential security breaches.
Addressing the Challenge:
The security industry faces a persistent challenge: ensuring that security guards diligently follow their designated patrol routes, effectively safeguarding properties and personnel. Our prototype tackles this challenge head-on by providing a real-time, comprehensive solution for monitoring security guard patrols. Our system meticulously captures and records the security guard's every movement, providing a detailed record of their patrol path. This tracking capability enables immediate identification of any deviations from the established patrol route.
Innovation and Key Concepts:
Our prototype is a conventional security solutions by incorporating features and adopting an innovative approach. At the core of our system, a GPS tracking technology that continuously monitors the security guard's movements, providing precise information about their patrol route. This real-time tracking capability goes beyond just location monitoring; it captures the guard's pace, direction, and any pauses or deviations, creating a comprehensive picture of their patrol activities.
Complementing the GPS tracking system is a data analytics platform that transforms raw patrol data into valuable insights on a cartopy map. Our platform analyzes patrol patterns, identifying areas of frequent coverage, potential gaps in security, and any unusual deviations from established routes. These insights empower security managers to optimize patrol routes, ensuring that every area receives the necessary attention.
- Slides containing: Problem Statements, Elevator Pitch and Potential issues -
- Updated Product Box -
- Updated Technical Solutions Map -
- Updated Technical Solutions Explanation -
Firstly, the ESP32 Nodemcu with its built-in WiFi capabilities can effectively communicate with web servers and send data. In this scenario, the ESP32 will be used to collect the GPS coordinates, which will then be transmitted to a MySQL database via an HTTP POST request to the PHP Webpage. PHP, a scripting language, will handle the server-side processing of receiving the data from the ESP32, and storing it in the MySQL database. Next, a python script with multiple libraries such as matplotlib and cartopy will be used to plot out the entire route and pathway of the security guard to visualise for areas of improvements.
After the valuable experience and knowledge learnt about requiring the Raspberry Pi to provide date and time for the Neo M6 GPS Hat. I decided to try and fix the problems that was unresolved back in the hackathon week.
Since the date and time information can be obtainable from other sources, I prioritized in troubleshooting and the retrieval of GPS location coordinates. Previously in the hackathon, I was successful in getting the NMEA (National Marine Electronics Association) messages from the GPS module to display on the Arduino IDE's serial monitor. However, despite using various GPS libraries like TinyGPS to parse the GPS data, the serial monitor kept showing that the location was not available, even when the device was under the sun with zero obstructions in the way.
After looking through various information about GPS modules, testing different GPS libraries, and checking for any missing inputs in my C++ code. The GPS coordinates still remained unretrievable. As a result, I made the decision to acquire another Neo 6M GPS Module without the Raspberry Pi hat attached to it. Surprisingly, after connecting the new GPS module to the ESP32 Nodemcu, it functioned properly without requiring any modifications to the C++ code. Therefore, suspecting a potential fault in the GPS module leading to the absence of not providing GPS coordinates.
- Unavailable GPS readings from serial output -
- New Neo M6 GPS connected to ESP32 -
- Captured GPS coordinates from serial output with new GPS Module -
Considering the potential inaccuracy of altitude readings, I intend to enhance the precision by incorporating a barometric pressure sensor for indoor altitude measurements.
For the start of a new term in week 12, I have created a setup file for a database that consists of three distinct tables: Admin login, Security guard info, and Records for storing GPS coordinates. The database schema has been implemented in a MySQL database hosted on an XAMPP server. Furthermore, Apache has been used to host a webpage for the Seer application. Thus, allowing the ESP32 to communicate with the web server by sending an HTTP POST request to a PHP script, which subsequently adds the data to the database.
Besides that, two php files are created. One of the two php files contain the database connection to MySQL while the other php file checks for any incoming HTTP Post request. If so, it creates an sql statement to insert the incoming data that was in JSON (Java Script Object Notation) format converted to a php array before inserting to the database records.
- PHP Database Connection -
- PHP Insert data in MySql Database -
For the database creation, the admin login table has a primary key that automatically increments, as well as fields to store the email and password for admin login credentials.
The security guard table contains a primary key that automatically increments as security guard id and a field to store the name of each security guard.
In the records table, there is a primary key that automatically increments, along with the fields to store the security guard ID, record time, altitude, latitude, and longitude of the captured geographic coordinates. The security guard ID field in the records table is a foreign key that references the security guard table.
- Database Setup Script -
The additional codes added to the ESP32 Nodemcu includes, declaring the address of the Apache server to the PHP file, allowing it to post the captured GPS coordinates. Before sending out the JSON string that contains information such as the security guard ID, record time, altitude, latitude, and longitude, the ESP32 first checks for a WiFi connection to ensure validity of the GPS coordinates. The information is then sent via an HTTP POST request to the PHP file, which inserts the data into the database records.
- Declare Apache Server to send http request via a php file -
- ESP32 code to send GEO coordinates in JSON via HTTP Post -
Moving onto week 13, I received the GY-63 MS5611 Barometric pressure sensor module to capture altitude indoors. The BMP sensor module is a high precision pressure sensor and comes with a built-in 24-bit ADC and can communicate with the ESP32 via I2C. By ulitizing the I2C communication protocol, the ESP32 can communicate with the BMP sensor module by sending and receiving data via the SDA and SCL pins.
The altitude provided by the BMP sensor module is more accurate than the GPS module, which is prone to errors due to the presence of obstacles and interference from the surrounding environment. As a result, the BMP sensor module is a more reliable source of altitude data for indoor use. Therefore, allowing the benefit of the altitude data to estimate the floor that a person is currently located at.
Althought the BMP sensor module is capable of providing altitude data, it is unable to provide the exact floor location. This is because the altitude data is relative to the sea level, which is not the same as the floor level. Hence, the altitude data must be converted to the floor level before it can be used to estimate the floor location.
Besides that, there are also additional factors to include such as elevated ground.
For example, if a building is located on elevated ground, the altitude data is higher than the actual floor level which affects the altitude data. Thus, it must be adjusted to account for the elevation of the ground.
The estimated floor does not match the actual floor level from my house due to the small elevated ground. Therefore, i am currently working on refining the code to account for the elevation of the ground in the next few weeks.
- Connected GPS module & BMP module together with ESP32 -
Due to the lack of male to male jumper wires, the placement of ESP32 Nodemcu, GPS and BMP modules on the breadboard is not possible at the moment. Thus, with some male to male & female to female jumper wires, the different components are temporarily connected together.
- Updated ESP32 code with additional BMP readings -
- ESP32 output from both GPS & BMP -
- MySQL Database with BMP altitude and floor estimation data appended-
For week 14, to enhance security and enable tracking of the assigned security guard, a bluetooth feature is implemented on the ESP32 Nodemcu. This feature requires the security guard to establish a bluetooth connection with the ESP32 Nodemcu using their mobile phone before the seer tracking device can begin its pathway tracking.
This bluetooth feature will capture the bluetooth mac address of the security guard's mobile phone. Since bluetooth mac addresses are unique, it allows the benefit to be used for the identification of the security guard. The captured bluetooth mac address will be stored in the database records along with the GPS coordinates and altitude data.
However, after integrating the bluetooth code into the main GPS Tracker, an error occurred due to the storage space being exceeded. This was caused by multiple libraries with initialised objects that consumed a significant amount of storage space.
After some online research to find solutions for this issue, I came across some potential solutions that involves using older versions of the libraries to reduce storage space. However, implementing these older versions may result in the loss of some functionalities. As a result, I have decided to test and explore alternative solutions in the following weeks.
In week 15, I discovered a more effective solution to address the storage space problem with the ESP32 Nodemcu without the need to downgrade the verions of libraries that risk the chances of some loss functionalities.
Switching the board selection from DOIT ESP32 DEVKIT V1 to ESP32 Dev Module was necessary as it offers a larger flash memory size of 4MB, which is divided into 2MB for the program and 2MB for the SPIFFS file system. On the other hand, the DOIT ESP32 DEVKIT V1 only provides 1.25 MB of program storage space and 0.3125 MB of dynamic memory (RAM).
- ESP32 Board Selection -
By modifying the partition scheme that dictates the allocation of memory on the ESP32, I was able to allocate the available 4MB of flash memory more efficiently. By default, the partition scheme assigns 1.2MB to the program and 1.5MB to the SPIFFS file system. However, this configuration doesn't offer sufficient space for the program to function optimally. To overcome this limitation, I adjusted the partition scheme to allocate 2MB for the program and 2MB for the SPIFFS file system, which successfully resolved the storage space issue.
Besides that, I updated the database setup script to include the bluetooth mac address field in the records table with a character length of 17 that is not null. Mobile phone's bluetooth mac address is 17 characters long, inclusive of semicolons.
I also successfully integrated the bluetooth feature into the main SEER GPS Tracker code. The ESP32 Nodemcu is now able to establish a bluetooth connection with the security guard's mobile phone and capture the bluetooth mac address. The captured bluetooth mac address is then stored in the database records along with the GPS coordinates and altitude data. Should the bluetooth connection be lost, the ESP32 Nodemcu will only continue tracking the pathway once the bluetooth connection is restablished.
- Update DB Script with Bluetooth -
- ESP32 Bluetooth Function Integration to main SEER GPS Tracker -
- ESP32 SEER GPS Tracker Output -
Lastly, I obtained new jumper wires to allow better fiting to the breadboard. The GPS and BMP modules are both placed on the breadboard and connected to the ESP32 Nodemcu via male to male and female to female jumper wires.
Previously, i have referenced the elevation / altitude above sea level around Ngee Ann Polytechnic from the website (Free Map Tools) to obtain the altitude data, the average elevation altitude around Ngee Ann Polytechnic is about 34.8 meters.
Using this reference point, the altitude from ground is calculated by subtracting the altitude above sea level from the altitude from the BMP sensor module. The altitude from ground is then divided by 3.5 meters, which is the average floor height of a building. The result is then rounded up to the nearest integer to obtain the estimated floor level. If the altitude falls into a negative value, the estimated floor level is set to 1.
On week 16, I decided to verify the accuracy of these readings by comparing them to the actual floor level near the Convention Centre stairs. However, the accuracy of the altitude readings from the BMP sensor module shifts depending on external factors just such weather conditions. Thus, affecting the accuracy of the estimated floor level. As a result, I made temporarily adjustments to set a temporarily reference point to test if the barometric pressure sensor could detect altitude changes from stairs.
In addition to that, I conducted further altitude testing in Blk 27 / 31 to validate the precision of the altitude readings and floor estimation. As the ground elevation in Blk 27 / 31 differs from Convention Centre, I re-adjusted the reference point to the ground altitude readings of Blk 27 / 31, which measured approximately 27.09 meters after conducting an altitude survey by walking around the ground level area near the lift. By utilising the ground reference point, the estimated floor is calculated by subtracting the ground altitude from the current altitude followed by dividing the height difference between each floor. This time, the estimated floor was accurate and aligned with the actual floor level.
The main reason why the altitude fluctates near the Convention Centre stairs is due to the change in air pressure. The BMP sensor module measures the air pressure, which is used to calculate the altitude. In open spaces, the sensor is exposed to direct sunlight, wind, and other environmental conditions that can cause fluctuations in atmospheric pressure. On the other hand, a shaded area near the lift of Blk 27 / 31 experiences more stable atmospheric conditions due to reduced exposure to direct sunlight and wind. As a result, the altitude readings are more consistent and accurate.
- Testing Altitude readings in Blk 27 / 31 -
*Images are cropped to the best fit that shows both the floor level and the Serial monitor from the laptop*
*Right click and open these images in new tab to zoom into Arduino IDE Serial monitor to verify altitude & floor estimation readings*
- Updated Technical Solution for SEER GPS Tracker -
The technologies implemented in the SEER GPS tracker is an ESP32 NodeMCU that is connected to both the GPS Tracker device and Barometric pressure sensor to capture and collect GEO location data and altitude readings. With the ESP32 NodeMCU's built-in wifi and bluetooth capabilities, it can effectively ensure identity of the security guard that is connected to the SEER GPS Tracker via bluetooth from their phones. And with wifi connection, it can send data through a HTTP POST to the php script residing on the apache server. PHP will then handle the server-side processing of receiving the data from ESP32 and storing it into mysql database.
Next, pyscript and flask is utilized ensure that the python script with multiple libraries such as matplotlib and cartopy, could be used on the web application that is the primary source to display and plot out the entire route and pathway of the security guard to visualise for potential blind spot areas that is not covered.
In week 17, I took part in a technical assessment that was given to us to assess and gauge our understanding of the technologies and concepts that was learnt. I went in without any prior preparation in order to thoroughly evaluate my performance and assess myself fully.
After completing all the assessments, I realised that I had become somewhat rusty on some concepts and knowledge learnt. The results of the assessment did not meet my expectations and slightly disappointed with my performance on the CSS portion of the assessment. However, I was able to identify the areas that I was weak in and needed to improve on.
The key takeaway from this experience is the importance of revisiting previously learned knowledge and concepts in order to retain information effectively.
