This article is the second feature in the Projects of UpCode Series I, where UpCode Academy instructors lead teams of graduates to work on (hopefully) fun projects. You can find the first one here.
Interested to learn how to build projects like this? This project is built by our graduates! Have a look at our Python Development course for beginners and Data Science Introduction course for engineers trying to start a career in Data Science.
Who says Singapore is not an exciting place? A cursory search on events listing sites Peatix, Meetup, and Eventbrite shows that there are dozens and dozens of events happening each day.
Given that there are so many kinds of events happening out there, it is a pain to search for the right events that suit your interests. Furthermore, you’d have to go through multiple events sites to find events that interest you.
What if we could gather all of these events into a single listings using an events aggregator for easy browsing? That’s what UpCode Academy Data Science graduates Wenkai, Terry, and Keli thought to do. Advised by instructor Jackie, the students came up with a website that aggregated events based on a particular category for users.
Before embarking on this project, there were several considerations to make. Firstly, we must complete the project within 2 months. Secondly, we had to use most of the things we learned from UpCode Academy classes.
Exploratory assessment and feasibility
When it comes to listings or aggregation, regardless of industries or verticals, the first thing is to check for any available APIs.
APIs, or Application Programming Interfaces, allow applications, be it web or mobile, to communicate with one another formally. APIs allow you to draw as much or as little as you need from a company’s servers.
An analogy would be someone having a meal in a restaurant. You check the menu (choose what API calls to make) and tell the waiter (the API) what your order is (what kind of data you want to retrieve from the server). The waiter then takes your order to the kitchen (database), retrieves the food (data), and delivers it to you.
Given our idea, we performed a search for APIs for the three main events platform: 1) Eventbrite, 2) Peatix, 3) Meetup. We found that Eventbrite and Meetup had well documented APIs, whereas Peatix was completely closed off. This meant that we would have to retrieve the information from Peatix in a different manner.
Event Advisor Workflow
We decided to call our events aggregator Event Advisor. As mentioned, while Eventbrite and Meetup had APIs, Peatix was not as friendly in terms of retrieving events. As such, we used Selenium to query Peatix and retrieved the events programmatically. With reference to the image above, this is the sequence of things that we planned our product, Event Advisor, to do:
- User makes a query which sends a request to our Event Advisor server
- The Event Advisor server makes API calls to the event site database
- Event site returns a list of events to the Event Advisor server
- The server also programmatically queries Peatix for events
- Event Advisor then retrieves the events from Peatix
- Event Advisor collates all of the events obtained and returns it to the user’s browser
We started with the Bootstrap 4 template for the front-end of the site (1) because we did not want to spend too much time on the front end of Event Advisor. As for the backend, we set up a Flask server to perform the rest of the operations (2-5).
In addition, we opted for preset categories to standardize our API calls. This was because Eventbrite had 21 overlapping categories whereas Meetup had 24 unique categories.
Choice of information shown
When users select a particular category, Event Advisor retrieves the event details from Eventbrite and Meetup via API calls, and then the server programmatically queries Peatix for events in the chosen category. The server then selects and displays the following details in a list:
- Event source, i.e. is it from Eventbrite, Meetup, or Peatix
- Event name
- Start date
We chose to display the minimum number of information because not all kinds of information can be found across these three platforms. If we were to plot a Venn diagram of information and features each site provides we would observe this:
The Venn diagram shows the information that the three platforms have along with the overlapping information in the intersections. We excluded the header images because that would increase our load times.
When the user loads the events aggregator (give it some time to load, we’ll explain why later), the user will be greeted with a dropdown menu containing the 15 chosen preset categories, ranging from Arts to Sports.
Once the user chooses a category and click “Search”, they will be arrive at a page containing a list of events. Users can click on the individual event names to go to the original sites for more information.
Before we hosted our server on the cloud using Heroku, we would use our own computer as server first for testing. This led to load times of over 12 seconds. However, after extensive optimization and hosting it on Heroku, this brought the loading time down to 6 seconds which was more acceptable in today’s website standards.
The reason for the long wait time is because of Peatix. More specifically, the Flask server that we set up is synchronous, i.e. it has to run code sequentially. The Flask server has to spend time retrieving information from Peatix first, organize the events’ details, before finally combining them with the other two sites’ events.
Furthermore, we used the free tier of Heroku (the hosting site) which compromises speed and scalability. The server goes to sleep every 30 minutes which means when you access it for the first time in a while, it’ll take a bit of time to warm up before working.
As a prototype, however, we reasoned that Event Advisor does what it was advertised to do. We successfully came up with an idea, designed the processes, and implemented them using what we learned from class.
We have created, to the best of our knowledge, the first automated events aggregator and listings in Singapore. Once again, check out the excellent work here (best viewed on the desktop and give it some time to load), and the Github repository here.
If you want to build projects like this, you can do so by attending our Python Development course for beginners and Data Science Introduction course for engineers trying to start a career in Data Science.