Category Archives: Database

Managing Relations with Elasticsearch

1. Introduction Elasticsearch is a search engine. It provides a distributed full-text search engine over schema-free JSON documents. As we all know Elasticsearch is a great product to index and search through a large number of documents. It supports various functionality like term and range queries, full-text search and aggregations on large data sets that are very fast and powerful. It is built over Lucene which is a high-performance, full-featured text search engine library written in Java. When we are indexing data, the world is rarely simple as an independent data existing in... Read More

Tip-toe through Naukri, India’s No.1 job site #OneDayTale

Glipmse of Hackathon 2016. We love open source and hence we are very open to transparency. The inception to this blog post was an idea to share the walk through of how a user is served in a typical day by your beloved Naukri.com. Naukri is a black box which gives you results but under the hood it is way more than what anyone can think. Under the hood [caption id="" align="alignnone" width="800"] Under the hood working of a car, GIF image.[/caption] Source Something like below is what we are continuously doing to serve you better. [caption id="" align="alignnone" width="549"] Car engine... Read More

MySQL Installation/Configuration Automation

MySQL Installation Automation In today’s fast paced technology world, where numbers are growing be it data size, no. of requests per second, no of databases etc., to manage these big volumes, standard operation procedures play a key role. It helps not only in ensuring smooth operations of web based products but also plays a major role for administrators to manage it in a controlled fashion. Standards are important, but enforcing them if one have multiple human resources to work towards achieving them is difficult. Configuring the setups manually is error prone and not manageable as your no. of installations grow... Read More

MongoDB Cluster Exceptions

In the previous blog for Mongo, we came to a conclusion how Mongo was a better choice for our situation. Carrying it forward we implemented Mongo with clusters to log the fields' edit and view time of the users. Problem Statement :- After turning live, we faced an unusual exception from the Mongo cluster which set up an alarming situation for us. The exception that we encountered was something like this :- Failure to send “getnonce” command to database “db1” . Failure to send all requested bytes So, getnonce is a command used by client libraries to generate a one-time password... Read More

User Data Logging Using Mongo

Torture the data, and it will confess to anything......... Why bother capturing data ? Data has become the most valuable commodity to every Internet company out there and therefore collection of data turns up to be a very essential task. Such was the need encountered by Naukrigulf team. The product team asked us to come up with the solution to log the modified and view timestamp for every jobseeker fields like username, salary, photo, work experience, etc. So, we needed to capture when a particular jobseeker viewed a particular field and which ones he edited. The very sole purpose of... Read More

DNS Resolution in PERL

DNS Resolver in Perl DNS, which stands for domain name system, controls your domain name's website and email settings. When visitors go to your domain name, its DNS settings control which company's server it reaches out to. Benefits of DNS: Domain Names are alphabetic, they're easier to remember. Capable of providing security. Companies that make use of DNS server are able to protect the company related data from being accessed by unwanted people. DNS servers are quite flexible to use. A single DNS server is capable of managing multiple Internet sub domains and domains. How DNS resolver... Read More

Database Backups Using MySQL-ZRM

Data is one of the biggest assets for any organization, so maintaining multiple copies of updated data and to make it available in case of any hardware/software failure is a key thing to keep the business running. Thus, an efficient data backup solution is a must. Taking backups of huge data is always a challenge because having a hot backup of TBs of data is impossible and backing up the complete filesystem doesn’t guarantee data consistency. While exploring different open source backup solutions, we came across MySQL-ZRM (MySQL Zmanda Recovery Manager) which matched our requirements to an... Read More

Running optimization on MyISAM Engine truncated the whole table !

Web development is always full of surprises whether you are developing website, maintaining databases or trying to save yourself from hack attempts.Recently We encountered a strange behavior of MyISAM engine in mysql5.4+ version when We tried to optimized the table containing around 50GB of data but it truncated showing this . mysql> select count(1) from templates; +----------+ | count(1) | +----------+ | 0 | +----------+ 1 row in set (0.00 sec) We lost almost 53GB data of templates which were uploaded by users. Luckily we had backup of that mysql instance, so we... Read More

MyTrend for MySQL

Data – the heart of any website. MySQL – most websites (including ours) are powered by it. With increasing transactions, servers and database sizes, it was becoming a pain to manage and monitor all of it. We thus built MyTrend – an ingenious tool to monitor data growth trend and server space utilization simple enough to be customized easily. This is what it does :  * Monitor Everything - Everything inside your MySQL can be monitored (Database Size, Connections, Slow Queries etc)  * Analyze MySQL data collected over a period of time  * Capacity Planning - Plan the growth... Read More