We have been using Elasticsearch as the storage and analysis tool for our centralized logging in addition to various full text search needs in different applications. The cluster being used for centralized logs handles a heavy load, indexing nearly 1.5 billion documents a day and going to about 50k documents per second during peak hours. The same cluster bears the load for searches to handle alerts and provide data to kibana dashboards. Hence the cluster performs large amount of aggregation queries in addition to document indexing. With heavy loads the cluster started having performance... Read More

Logs are important part of any system as they give deep insight into what is happening with the system. They also helps in figuring out what went wrong when something unexpected happens.  Most applications generates logs in one form or the other and they are generally written into files on the local disk. A web application consists of various components and each of them generates logs. Few of them are mentioned below. Access logs from web servers like Nginx, Apache Logs from back end applications (Java, PHP, python etc.) Logs from Database, Cache system, Queues etc. System... Read More

The Symfony Framework is well-known for being really flexible and is used to build micro-sites, enterprise applications that handle billions of connections and even as the basis for other frameworks. Some time back we started using Symfony2 for our new projects. Symfony2 comes with twig as the default templating engine but we chose to use smarty as the templating engine using the SmartyBundle of Symfony2. Smarty is great, but for maintainability reasons, they have deprecated the possibility of adding php tags in the template. This means we will not be able to use php class constants... Read More