Picoo “Innovating Play” is the first game console to be played outside. It’s meant for children and adults to play outdoors in a group. Each player has one device, which can scan the game cards and synchronize with nearby Picoos to play the game together.

Fourtress is closely involved in the embedded software development (including device drivers) of parts of the software for the game controller. The challenge is to make it a robust game controller that can take a beating. In addition, it must work anytime and anywhere and… There is no screen!

Techniques and tooling that are involved include RFID chips, firmware, device drivers and battery management.

Diagnoptics is a pioneer in and inventor of advanced diagnoses equipment, with which the risk of diabetes and associated complications can be diagnosed and estimated in a non-invasive manner.

We have developed multiple Android and iOS apps for this client, all of whom must work closely with Diagnoptics AGE-scanner. The AGE-scanner non-invasively measures the tissue level of Advanced Glycation End products (AGEs). The scanner communicates all the data to the apps using Bluetooth Low Energy (BLE). The challenge for Fourtress is to correctly set-up the BLE-communication in which data such as measurement results, battery level and device info are real-time processed in the apps and shown to the users in a clear way.

Three of the in total four apps that we have developed for Diagnoptics use this data to inform the user. The SkinAge app can translate the measurement data into an easy to understand SkinAge result that compares your calendar age with the aging of your skin. The MyAgeNow app can translate the measurement data into an easy to understand result that shows your level of glycation and compares this with other people of the same age. The latest addition to this list of apps is the DiabSpot app. This app uses the users BMI, age and measurement data to give an indication of the risk on having Diabetes type 2. All results of these apps can be saved and shared directly.

Besides these apps there is the Diagnoptics Service App. The Diagnoptics Service app enables the user to do an “over-the-air” firmware update on the scanners. This way, the scanners can always run on the latest firmware without having to be sent back to Diagnoptics. The app will compare the current firmware version of the scanner with the latest available on the server. In the case of a newer version being available, the app will temporarily store the firmware as a ZIP-file on the phone so the app can install it on the scanner, again via BLE.

Because of these apps and the scanner it has become very easy to get an indication of your AGE health. A quick scan and the apps can tell you exactly how you are doing.

From care assistant to lifesaver: raising a drone with added social value

Foto: Bram Saeys TU/e Studententeam Blue Jay

“I may be more experienced, but these young people come with fresh ideas. We learn from each other.”

Take a team of smart students from the Eindhoven University of Technology (TU/e) with a vision to develop an autonomous drone that adds real value in healthcare and related fields, make them aware of the ever-present threat of children drowning in swimming pools and you’ve got a challenge to get passionate about.

Despite the best efforts of thousands of well-trained lifesaving professionals and volunteers, on average each year 60 people will suffer injuries as a result of almost drowning in Dutch swimming pools, and 2 will actually drown. By far the most common group of victims is children.

According to Anja de Brouwer, a long-serving volunteer at the ‘Eindhovense Reddingsbrigade’ (the life rescue volunteers service in Eindhoven), who is trained to realistically simulate accident victims, “When a child’s drowning they usually don’t shout or wave their arms as you’d expect. It just goes quiet and, before you know it, you’ve lost them.”

Serious roles

Anja is one of the many volunteers who recently took part in a large-scale event at the Pieter van den Hoogenband Swimming Stadium organized by the rescue service to help the Blue Jay Eindhoven team develop a drone that can assist lifeguards by performing three key tasks: monitoring activity in a busy pool, detecting a swimmer in distress and responding appropriately.

During the evening, eight cameras continuously film the volunteers as they simulate drowning people. “To learn how to detect which movements mean someone may be in trouble rather than playing, the drone must be shown lots of different videos of drowning people,” explains Liza Boormans, the Blue Jay team manager. “Once it has enough data, it can recognize and detect drowning behavior itself.” Thankfully, film footage of people really drowning is rare. Hence the need to film the LOTUS volunteers role-playing victims.

Not waving but drowning

This process of detecting someone drowning is one of the real innovations of the lifeguard assistant project, and one of its biggest challenges. “There are already detection methods on the market that spot if someone has drowned, but the Blue Jay team is developing a drone to detect if someone is about to drown — in time to help them. That’s a much more complex challenge,” says Arjen van de Wetering, an Internet of Things (IoT) consultant at IBM who has been supporting the Blue Jay team since IBM became a main partner to the project a few months back.

“Distinguishing between play and distress is difficult using traditional analytics methods” explains Arjen. “So the team is using artificial intelligence (AI), employing IBM’s Watson technology. The team will use the same drowning simulation videos to train the system and to test whether it can make the drowning-playing distinction, and later whether it can make that judgement effectively in real-life situations.”

This detection process isn’t simple. Having singled out individuals from its aerial view of the whole pool, the system must follow people using approximately 2 seconds of consecutive video frames to determine how someone is moving. The oldest frame from those 2 seconds is then dropped and replaced by a new one. The system is thus continuously analyzing how people move within a 2-second timeframe. The calculations to determine whether a swimmer’s movements constitute playing or drowning take a huge amount of computer power. In our first pilot setup, all these videos are streamed live from the cameras to the cloud, where the calculations are then done on IBM’s Watson AI mainframes. The ‘decision’ about the swimmer’s movements is then made by Watson and sent back in a matter of milliseconds to the drone, which acts accordingly — a classic IoT solution. For the final system, the team probably don’t want to depend on an unreliable internet connection, so it might consist of a trained Watson-based application on-premise. But that decision has yet to be taken.

International and multi-disciplined

Blue Jay Eindhoven is an on-going project with the ultimate goal of developing a drone that can assist professionals in healthcare settings. The team is made up of students from the Technical University of Eindhoven (TU/e) who take time out from their studies to join Blue Jay for a year.  “Because you get no elective points or financial support for working on Blue Jay,” explains Liza, “only people who are very committed to our goals and vision join the team.” Liza’s own role as team manager ends in September when she returns to her full-time studies in Medical Sciences & Technology.

The current team is an international group of 18 students from 9 different academic disciplines. As the Blue Jay personnel change pretty well entirely each academic year, continuity is assured through an advisory board that includes some of the previous year’s students who support the next year’s team. “The board ensures the team follow the long-term Blue Jay vision,” says Liza, “but gives them sufficient freedom to add their own insights and innovations.”

Last year’s team focused on developing a drone to assist in the Máxima Medical Center near Eindhoven. Much progress was made. It could, for example, play games with child patients. But a stumbling block, which has not yet been overcome, is noise: a drone is still pretty noisy and that obviously isn’t handy in a hospital environment.

The perfect setting

Ironic, then, that it was during a brainstorming session on solving the noise problem that the team heard about the case of a 9-year-old refugee girl who drowned in a swimming pool in Rhenen. They immediately saw the potential for a Blue Jay drone to help prevent such tragedies. “Swimming pools are a perfect setting,” explains Liza, “Noise isn’t an issue, but we can still demonstrate the huge potential value of the drone: the ideal proof of concept.”

But as team project manager, Mike van Sighem, points out, the swimming pool drone has its own challenges. “As well as drowning detection, it must be able to autonomously navigate buildings, notify personnel, and bring bulky and heavy objects to an accident scene.”

Pim van Dommelen, Blue Jay’s mechanical engineer, sees this last as probably the biggest mechanical challenge of the lifeguard drone. “It needs to be light, otherwise you have to switch batteries every few minutes, and a drone that’s out of the air too often isn’t much use as a lifeguard assistant. But at the same time, it must be powerful enough to fetch heavy objects such a lifebuoy, defibrillator or first aid kit.”

Partners make perfect

NXP, the second main project partner, is helping the team with making the drone both robust and responsive enough to do its job. “Last year we helped the Blue Jay team develop a robust communication channel between ground station and drone,” explains Gino Knubben, Principal System Engineer at NXP and closely involved with the company’s support to Blue Jay. “This year they’re using a slightly newer version of the same technology, but our role is even more drone-focused, looking mainly at two major systems. First, the flight control system, which deals with the drone’s movement, orientation, balance and remote control – the team chose this year to switch to our flight controller, NXPhlite. And secondly, the application processor, which deals with the display and the interaction.”

For the project, functionality is critical and one big benefit of working with world-class companies like NXP, IBM and Fourtress, the third and final main Blue Jay sponsor, is the Blue Jay team’s access to advanced technology not found in consumer products. “Compared to regular Wi-Fi on your phone,” explains Gino, “our automotive-qualified connectivity solutions are superior in robustness, RF reflections, power consumption and the number of ad-hoc connections you can do per second.” The robust connections come from 802.11p technology used in the automotive industry, and can handle the drone’s rapid movement and ad-hoc communications. Gino laughs, “Two years ago, everybody had to remember to switch off the Wi-Fi on their phones before testing or demonstrating. Now with 802.11p we’ve resolved that particular interference issue.”

One-stop shop

Another benefit of having NXP as a partner is that, unlike many semiconductor manufacturers, they can provide a wide range of components. “We’re providing a large part of the electronics they need for drone development,” says Gino. “Not just the wireless link but also high-performance microcontrollers; proven reference boards with all the components assembled; and even complete sub-systems, including supported Operating Systems. The team then adds their peripherals, displays, sensors, applications, graphical user interfaces, etc.”

But Gino is first to say the collaboration is not one-way traffic. “The Blue Jay team are basically beta-testing some of this technology for us and their feedback is invaluable. For example, on development of the FMU (Flight Management Unit): along with all the other partners involved, they provide feedback, they test… It’s outside the students’ comfort zone and the remit of the project. So that wider commitment is great. They also visit us at NXP on a regular basis, which is a lot of fun.”

And there are areas where old hands learn from the students. “They’re very knowledgeable, for example, on the ‘total drone’, such as how to calibrate and configure it. And have some seriously skilled drone pilots! Another example is hardware assembly experience. We’d been assembling drones at NXP ourselves, which took a lot of time. Then Jari van Ewijk, electrical engineer from Blue Jay turned up and built one in under 90 minutes. I asked how he could do it so quickly. He said ‘how many prototypes you think I’ve had to build? We’ve crashed quite of few of them…’ He could probably have done it blindfold! The next day we received pictures and movie clips of their first flight with that NXP-drone kit!”

Another spin-off from Blue Jay is that the technology involved can have applications beyond the narrow (albeit very important) fields of lifesaving and hospitals. “For example, there’s huge potential for FMU technology and drones in areas like warehouse logistics,” says Gino. “So the beta-testing also contributes to making this a proven solution NXP can offer in other contexts and sectors where the same parameters apply.”

Priceless know-how

Fourtress provide software solutions for technical systems. “We’ve been involved since the first Blue Jay team,” says field manager, Douwe Gerritsen. “Initially, we simply provided technical knowledge. But as we have engineers with pretty strong drone-related experience, for the past two years instead of money we have loaned the team an engineer for 3 months.”

It’s win-win as the engineer also gets interesting experience. He has worked on the low-level systems, Lidar, obstacle detection, the drone’s inner workings, the ultra-wide band and data communication systems. Fourtress also provided a student on placement with them to work on the Blue Jay mobile app development and drone-API as their thesis project.

“Probably our biggest challenge in developing the app is that Blue Jay team projects are very dynamic,” says Douwe. “They set ambitious goals. That’s inspiring, of course, but means implementation tends to change along the way as they encounter problems. You try to work with that flow, so you don’t become an obstacle yourself. But sometimes you need to help them set boundaries in terms of what’s achievable within the timescale.

“There’s also the technical challenge of trying to fly a drone in populated environments, including the paradox of wanting a drone that both works totally autonomously, but is also controlled by people.” This means a person must be able to use the app to give the drone commands, after which the drone figures out for itself how to execute the task. Sometimes deciding autonomously, for example, which route to take or task to prioritize. But the interface needs to let people easily set new tasks and get a clear overview that updates them on what each drone is doing at that moment.

“The app is basically the drone’s manager. It gives the drone tasks, but the drone decides how to execute them. The prioritization aspect is interesting. It’s important the user can schedule tasks to be done later or immediately. But also, that they get feedback on how quickly the drone will actually do the task. Because when an emergency happens it will override everything else. This variety of tasks makes things a lot more complex.”

Another challenge is notifications. “The tricky part is that people will use their phones not just to control the drone, but also to surf the internet, take calls, etc. Technically, this means the app must be able to run a service in the background even when it’s not active, so it can still give users notifications, for example of an emergency or a completed task, when their drone app isn’t active.

Douwe sees the Blue Jay partnership as an invaluable one for Fourtress. “How we operate with Blue Jay is a bit of a model for how we try to operate with all our customers. In that, if you hire one Fourtress person, you hire all of Fourtress: all our knowledge is at your disposal.”

Professional fans

Blue Jay gets students out of their ivory tower and working with real-world organisations to address real-world needs. So it’s reassuring when society responds so positively. “The Reddingsbrigade were very enthusiastic from the start,” recalls Mike. “It’s cool how they’re so open to adopting new technology. Ultimately, we share the same goals. By working together, we can help each other achieve those goals better and quicker.”

At IBM, Arjen has quickly become a huge fan of the students and what they’re doing. “Like much of my own work, this project is at the forefront of technology. I bring my experience to the project, but these young people have a different perspective and come with fresh ideas, so we definitely learn from each other.” Douwe echoes this view on behalf of Fourtress. “Because with Blue Jay you’re working with such enthusiastic and dedicated people, it makes it so much fun.”

At NXP, Gino is particularly impressed by how the Blue Jay team leverage maximum value from partners’ expertise and function so maturely as a unit. “They take technology from us for the drone, from IBM for in the cloud, use other partners for the mechanical chassis… operating just like an efficient little company, each with their own areas of responsibility. Of 18 team members, I only deal with 2 or 3 hardware engineering, electronics and software people. For young, inexperienced students it’s pretty impressive.”

With her hands-on experience, LOTUS’ Anja is equally excited by the drone’s potential. “It’s basically an extra pair of eyes. Even the best lifeguard gets tired, but not the drone. It stays alert, and that means once it’s fully operational it will help ensure people are safer.”

Those days are still a way off. For now, it’s all about proof of concept. Even so, the team will continue to work on a drone that can provide first aid in emergency situations. Because, as Arjen says, “if one day the drone saves just one life, the whole project will have been worth it.” And you don’t need AI to detect the truth in that statement.

Learn more about Blue Jay Eindhoven: https://www.bluejayeindhoven.nl/

Or get in touch with the team if you’re interested in supporting their work, getting involved or becoming a partner. Send an email to: info@bluejayeindhoven.nl

Our client wanted to equip one of the products from its Robot Vacuum Cleaner (RVC) range with connectivity for indoor use. For this, the RVC had to be controllable using an app (iOS and Android) indoors and outdoors. The RVC has been expanded with a WiFi module, and both apps needed to be compliant with the new standards that the client applies to all its connected products.

For this purpose, our client developed so-called common components. These standard components are to be used as the basis for the company’s connected products. An RVC is one of the most complex connected products. There was no reference project, which meant that Fourtress had to develop several common components in close collaboration with the client. All communication between the app and the RVC runs through a cloud service.

Fourtress developed both apps (native iOS and Android) in close collaboration with several stakeholders. We set up two App teams (Eindhoven and Minsk) to develop the apps simultaneously. Deployment of an App team in Minsk made it possible to reduce costs significantly. The RVC and its apps have now been launched in parts of Asia and are available in various Asian app stores.

Following the development of both apps, Fourtress also took care of their maintenance and release management, and continues to do so. These developments provide a foundation for future variations on the RVC Series: watch this space!

Pentair is a ‘multinational diversified industrial company’ with over 27,000 employees worldwide. As part of an acquisition, the company took over Hoffman’s, renowned for its ‘content meter’, a CO2 measuring solution for breweries. In fact, every brewery in the world uses Haffmans’ products. This segment of the company focuses on breweries, whilst other parts provide components for mobile water filter installations. These may be used to provide a small city with clean drinking water in case of natural disaster, for example.

Pentair develops and manufactures a variety of products for the soft drink and beer industries in-house. Design, calculation models, mechanical engineering and production support and software development come together in the company’s R&D department combines. Fourtress is working on several products, such as the Keg Monitor, a control system for the cleaning of beer barrels. When a barrel returns to the brewery after being used in catering, it is cleaned and checks are carried out to determine whether the closure is still working properly. For this purpose, a barrel is equipped with various sensors (temperature, pressure, etc.) as well as an embedded system for storing data. This ‘measurement barrel’ is placed on the cleaning conveyor belt to collect data.

Another project carried out with Fourtress is the inline alcohol meter. ‘Inline’ indicated that the sensor is placed in a beer tube, unlike lab equipment that sits on a worktop. The sensor measures the alcohol percentage as a meter of beer runs through the line every second. This measurement is based on a laser shining through the beer onto a spectrometer. The spectrum is influenced by the different components in the beer, and the presence of alcohol causes a slight deviation in the signal.

Finally, Fourtress and Pentair jointly developed the turbidity meter: a checking system that can calculate the turbidity of liquids. Although these projects involve different instruments, they have a great deal in common: they are all embedded systems with a so-called ‘ARM Cortex core’. Each runs on freeRTOS with its own library of hardware drivers and application modules, all written in C. One of the things Fourtress was involved in is the generation of generic blocks for a hardware-specific driver with a common interface. As a result, we have a greater choice of hardware, testing is easier, and we promote code re-use.

Cyclists or scooters emerging from nowhere at a crossing point; all too often they present a sudden danger. Motorists will often see them too late, and the cyclists are frequently unaware of their own vulnerability. The BikeScout, a Heijmans product, makes crossing the road safer.

Heijmans developed an intelligent system that contributes to the safety of cyclists. BikeScout is ‘intelligent’ in the sense that it continuously determines the speed and distance of approaching cyclists using radar technology. Based on this information, the system calculates the time it will take for the cyclist to reach the vehicle and it warns the motorist of the approaching cyclist (s). This warning is given using LED indication on the road surface. BikeScout also offers the option of viewing data from bicycle passages, such as numerical data, average speed and maximum speed, using a web application.

First generation BikeScout software was not expandable and was only used for test and demo applications. In addition, setting of the detection area was very labour intensive and time-consuming, as adjustments were needed for local conditions. As a consequence, Heijmans could not roll out the product on a larger scale or adjust it ‘over the air’.

Fourtress renewed the BikeScout software from the ground up and increased its stability. For this, a new hardware platform was used, among other things, and a new detection algorithm was developed. This algorithm makes system configuration, installation and adjustment very easy, anywhere in the world. The product is, therefore, no longer in the prototype stage and is now offered installers in several locations (nationally and internationally). In this way, BikeScout has become a fully-fledged solution for Heijmans.

The software has been set up in a modular way, and as a result derivative products for measuring and monitoring flows of cyclists through urban areas are now available. More complicated crossings where cyclists and motorists meet can be made safer with a combination of these technologies.

A complete BikeScout system has now been installed at the offices of Fourtress in Eindhoven. As a result, cyclists can safely cross the roundabout at the Beemdstraat, Meerenakkerweg and Kasteellaan