Log Analysis and Data Extraction Challenge
Download our pre-compiled application for your desired platform (Windows, Linux, Mac). It’s a command line application that will continually write to a log file in the same directory the appplication is executed from.
The application takes two optional parameters:
- Int – random seed
- Log output filename – The default file name pattern is
qrt_data_extraction_analysis_DD_MM_YYYY_HHMMSS.log
Log entries will be automatically appended to this filename, and your application needs to process this file in as near to real-time as possible.
Our challenge to you is to develop a tool simplify the extraction and analysis of the log output by this and other applications.
Some of the things your analysis tool should do include:
- Analyse multiple log files in parallel
- Option to continuously analyse logs as they are updated
- Filtering for important information (entered by user) – extra points for enabling regex
- Highlighting of information (entered by user) – extra points for enabling regex
- Display time difference between two entered points (rough latency analysis) statistical summary of all the times
- Log will also contain statistical information which should be displayed / summarised in some form (graphical?)
- Being able to save and reload selected options / filters etc…
- Raising ‘alerts’ if a certain type message is found or a condition is met e.g. latency is above threshold for a given amount of time
- Extra Bonus Points and Special Prizes will be awarded for novel and exciting features - nothing is off limits!
Extra points for making the application work via a web browser, any messaging system or some kind of fruit.
Hidden within the log messages are lists of the QRT Team’s favourite movies. If you’ve implemented the features as described above then you should be able to spot these. There’s also three number sequences hidden in the output. The first team that can list all the movies and tell us what the number sequence is will win cool QRT merch!
Application Downloads⌗
These are pre-compiled binaries - if you need support for a platform not listed below, come and talk to us!
Extra prizes for spotting and reporting any bugs (QRT maintains sole discretion over what is an actual bug!)
These binaries are harmless, but we cannot guarantee they’re bug free. Make sure you have (tested!) backups of your machines if you’re going to run these - this is good practice for running any random binaries you found on the Internet!
Running the Application⌗
You will need to run this application from a Terminal or Command Prompt window. If you’re using a Mac or Linux, you may also need to make this file executable, you can do this from the Terminal with the following command:
chmod +x qrt_data_extraction.Linux_x86_64
If you are using a Mac, you will also need to remove the Quarantine bit from the file, using the following command:
xattr -d com.apple.quarantine qrt_data_extraction.MacOS
Once done, you can run the app: ./qrt_data_extraction.Linux_x86_64
The application itself does not output anything when its running, so do not be alarmed if you don’t see anything on the screen! You need to check the files are being created in the same directory, if they are, the app is running.