You are viewing our old blog site. For latest posts, please visit us at the new space. Follow our publication there to stay updated with tech articles, tutorials, events & more.

Implementing Thrift for services

0.00 avg. rating (0% score) - 0 votes

We at Naukri use a number of services at the backend. By writing backend services, we brought moduarity to our system. There are a number of services serving different purposes. These services help us in managing and scaling various parts of the site independently.

For example, when you search for a resume or view a resume on Naukri, that sends a request to the Naukri’s frontend apps. But those apps don’t know how to fetch the resume data, they talk to a separate resume service that does the job.

So what are the advantages? All the resume data can be managed by resume service and every app using resume data is free from its management, availability and scaling.

Typically Naukri is based on LAMP stack. Most of our client apps are written in PHP. But for server end code we decided to use Java because of its connection pooling and multithreading support, which was required to serve the kind of traffic we were planning on services and the SLAs we were required to meet.

Need for communication protocol

Since most of our services are for internal consumption we didn’t want to use a web server for app to service communication, because a web server was costly in terms of resource requirements and response times. We decided to use unix socket communication. Generally, there’s a lot of code that has to be written for client and server communication. We needed a tool, or a library, or a framework that can do this job for us. To add up, our high level goals were –

a) Transparent interaction between multiple programming languages.

b) Maintaining right balance between efficiency and ease of development.

Then comes Apache Thrift in the picture.

What is Thrift ?

As per Apache Thrift’s documentation – Thrift is a software framework for scalable cross-language services development. It combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, and OCaml.

In simple terms, Thrift allows an application written in one language to exchange data with an application written in another language. Technically speaking, Thrift is a Remote Procedure Call (RPC) framework that can be used to develop scalable cross-language services. Thrift is an interface definition language, i.e., you can only write interfaces or pure abstract classes by Thrift. The Thrift compiler can generate corresponding classes and interfaces for any particular language from the Thrift interface.

As a serialization platform Apache Thrift enables efficient cross language storage and retrieval of a wide range of data structures. As an RPC framework, Apache Thrift enables rapid development of complete cross language services with little more than a few lines of code.

Other options explored –

a) CORBA – Although the idea of CORBA is amazing but it is very complex(overdesigned and heavyweight) and has steep learning curve.

b) DCOM, COM+ – It is embraced mainly in windows client software.

c) Google’s protocol buffer – It’s a popular framework which does the same job. It is stable and well trusted but it did not officially support PHP.

Thrift Features –

There are several key benefits associated with using Apache Thrift to develop network services or perform cross language serialization tasks.

Full SOA Implementation – Apache Thrift supplies a complete SOA solution . With Thrift, services are described using a simple IDL syntax and complied to generate stub code used to connect clients and servers in a wide range of languages. So as in our case, we used the Thrift compiler to generate RPC code for our service in PHP and Java. The web team now uses code generated in their language of choice to call the functions offered by the service, exactly as if the functions were defined locally.

  • Simplicity – Thrift code is simple and approachable, free of unnecessary dependencies.
  • Performance – With its lightweight RPC servers, Thrift enables programmer to build high performance cross language services.
  • Reach – Apache Thrift supports a wide range of languages like C, Java, C#, Go on a range of platforms including Windows, iOS, Linux, Android and many other Unix-like systems. It supports HTTP[S], Webscoket and an array of web tech languages, including Perl, PHP, Python, Ruby and JavaScript, making it viable in web oriented environments.
  • Flexibility – Apache Thrift supports interface evolution i.e. it makes it possible to evolve interfaces without breaking interoperability with modules built around older versions of the interface.

Challenges faced while implementing Thrift –

a) Even with minor changes in Thrift IDL, we have to compile and generate new PHP and Java code, which led to frequent changes on both client side and server side.

b) One major issue with Thrift 0.5 was with Enums. When we created enums in Thrift code, then while reading their value at PHP end, we received values as 0,1, 2, etc., not their corresponding string values we defined. So for this we have to customize some code in Thrift to get the desired value.

c) Generated code is not easier to understand, it has strange naming conventions and strange base classes.

PS: We are using thrift for quite sometime now and recently we moved to thrift 0.9.2 on our production environment, and it is working great for us till now.

One thought on “Implementing Thrift for services

Comments are closed.