We have been working hard behind the scenes to improve our product functionality and reliability, and are using this blog to share a little more detail about what's been happening, in line with a company goal of increasing visibility into how we manage problems behind the scenes.
You should expect to see more announcements here in the future, sharing some of the details of what we've been doing and what you can look forward to!
1. We have had some problems with our mobile application, primarily stemming from authentication issues under the hood. This occurred during an effort to improve our authentication solution by migrating to a new library to ensure the process of logging in and maintaining connections with our servers was faster; unfortunately this resulted in an occasional race condition that was not seen in internal testing but became evident when users started upgrading within the 4.2.x line. This bug proved quite difficult to find and fix, especially because it initially never occurred during our internal testing or with our group of trusted testers.
With release 4.2.5, available on both Android and iOS since early April, we are confident we have fixed this issue. We've also met the original goal of improving the reliability of connections, so users should see less disruption when trying to access their chargers and usage data. We have made some significant changes to our testing and app rollout procedures as a result of this experience, and are confident that this will help us improve the mobile app reliability in the future.
2. When any Hypervolt charger is used, the device records information about electrical consumption and sends it to our servers to be processed to optimise it for usage down the line. We save both the raw data as well as processed data, which is currently used by the mobile application and home dashboard for energy consumption information. The raw data helps us generate analytics for internal testing and investigation, some of which are included later on in this post.
When calculating charging sessions, we had difficulty determining the session start and end times when a charger was not connected to the internet and more than one charging session took place. The data processing pipeline we used would incorrectly combine these "offline" charging sessions and add them into the first "online" session we saw once the connection was restored.
When fixing this bug, we noticed another issue that would occasionally cause "online" sessions to be incorrectly aggregated, which would have a similar visible effect (an impossibly large charging session) but turned out to have an entirely different root cause due to the way incoming data was processed.
We have fixed both of these issues in our data ingestion pipeline, and are checking historical data to find and fix any instance where this occurred; this is being rolled out in the background as we have a very large amount of data to re-analyse for the fix.
2. When an electric vehicle is plugged into a smart charger (rather than just a regular socket), it uses the IEC 61851 standard to control charging. Each charger, and car manufacturer, uses a slightly different algorithm to determine what the charging rate should be. The chart below shows the data we recorded from a charging session where the negotiated limit (PWM, represented by the white line) jumped up to 32A and then rapidly dropped back down to around 7A.
We have now been able to identify and fix the mismatch, and the fix is included in a new firmware version being automatically rolled out across the charger fleet. In addition, we have now added these models to the simulations we use in firmware development to help us test future changes and ensure they work as desired across a broad range of vehicles.
4. We have different methods of accessing live real-time data and processed data from the application; these use different APIs as they have different infrastructure powering them behind the scenes. While we were attempting to make some performance optimisation to the realtime APIs, we unfortunately included a configuration change that took this service offline. While we were able to fix this with a fairly simple rollback, it took longer than it should have for us to detect due to shortcomings in our monitoring and automated alerting systems. We had already been aware of a need to do better in this area and had opened up hiring for a Site Reliability Engineer; our preferred candidate has now joined the company and will be leveraging their experience from Google and the FinTech space to make sure these systems are well-tuned and will help us solve problems before any users are affected!
5. The raw data we collect from Home 3.0 chargers, both in testing and installed in customer premises, is analysed internally to better understand how the chargers are performing in a variety of circumstances. While we have only just begun to make use of this, one early success has been an increased ability to detect potential problems and reach out to customers before the problem has any impact on their car or their charger. The chart below, from an internal analytics dashboard, shows an example where the detected grid voltage for an installed charger that occasionally gets over 240V - in this case, up to 253V!
Thank you again for being part of this journey with us - if you have any feedback, or suggestions for improvements, please let us know at feedback@hypervolt.co.uk
Leave Your Comment Here