Exploring “Elasticsearch”

What is Elasticsearch?

Elasticsearch is a platform for performing different kinds of searches and for analyzing real time data. It helps to organize data and make it easily accessible. It is almost similar to Database Management System. Elasticsearch is a cluster which contains multiple Indices(databases), which in turn contains multiple Types(tables). These types hold multiple Documents(rows), and each document has Properties(columns). Elasticsearch supports RESTful operations. That is, you can use HTTP methods (GET, POST, PUT, DELETE, etc) in combination with an HTTP URL to manipulate your data.

Status of tasks

So, moving back to my assigned tasks, I was working with elasticsearch indices. I was trying out many sample examples. But when I started using  my data, I got stuck with lots of errors. All my time were being lost in debugging. That is the saddest part of my progress 😦

I was stuck with one task for many days. But my mentors were very friendly. When I mentioned my issue in the last meeting, they readily agreed to help me anytime. That gave me great confidence. Now I’m trying out the options mentioned by them.

Issues

After installing Elasticsearch and Kibana, I started with adding data to elasticsearch.  My task was to add the raw perceval output and the threading info to elasticsearch. For putting data into elasticsearch, we need to create index. Following the tutorial, I was using curl method of adding data to elasticsearch. According to that, inorder to add large json file, BulkAPI method needs to be used. So, you need to add a header line as shown in the documentation. After adding the header line, you need to execute the following command to  add data to it.

curl -XPOST ‘yourhost:9200/jsonfilename/_bulk?pretty’ –data-binary @jsonfilename.json

At this point I was encountering errors.

error

So now, as mentioned by my mentors, I am learning to use sense in order to add data to elasticsearch.

Few more tasks

As all the platforms like elasticsearch, Kibana are new to me, its taking a long time for me to learn and start with these. But with the help of my mentors, I have started sorting out the issues and now I’m using different ways to resolve the issues. In the last meeting few more tasks were assigned :

  1.  Start working with matching script
  2.  Description of the process to run perceval

Along with this, the remaining tasks, which are yet to be completed, as I’m moving on very slowly.

 

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s