A C++17 Data Stream Processing Parallel Library for Multicores and GPUs
Web Dashboard
Starting from the release 2.8.8, WindFlow has its own web dashboard that can be used to monitor and profile the performance of running WindFlow applications. Once running, the dashboard opens a REST server on a port (8080 by default) of the machine. To avoid performance conflicts between the dashboard threads and the ones of the running WindFlow applications, the dashboard can be executed on a different machine with respect to the ones used to run the WindFlow applications to be monitored.
To use the dashboard, WindFlow applications must be compiled with the WF_TRACING_ENABLED macro. Other macros are available and can be used to specify the hostname and the port of the dashboard (by default the applications assume that the dashboard is executed on the localhost).
The main web page of the dashboard, with the summary of the monitored applications (running, terminated or interrupted for any reason) is shown aside. In the figure, the dashboard reports one running application while no terminated or interrupted applications are listed.
When the user runs a WindFlow application, it automatically tries to connect to the dashboard on a specific port. An additional monitoring thread is spawned by the WindFlow runtime to report statistics to the dashboard every second. The monitored application is added to the summary table of the dashboard.
By clicking on the application name, the user is directed to a separate web page where the statistics of the application can be inspected.
The user can check the monitoring information of each running operator and of each of its internal replicas. Statistics are updated every second, and report the number of received/transmitted inputs/outputs, processing time, and so forth. The figure on the left shows the report page of an application.
The last screenshot below shows the report of statistics opened for one operator and the historical statistics (input and output rate per second over the whole execution) for a different operator opened in the same web page. This can be useful to continuously track the application execution in order to discover bottlenecks in the dataflow and to undetstand how to optimize it as a further refinement of the development and configuration process applied by the developer.
The web dashboard aims at being a useful tool for developers of WindFlow applications. However, statistics are often intrusive and might slow down application performance (we measured a drop between 5 and 20% in our benchmarks). So, it must be used carefully.