Elastic Search
I got a chance to interact with Elasticsearch during my job & as I had to leave that job so couldn't continue to explore this area. But now I am trying to go through this via online tutorial on this & I see that in all such technologies or frameworks you need good amount of data to try different things.
And I dont have that huge amount data with me to explore these fields properly, but still I will try to put few things during my journey.
In the initial phase I don't see it much different than other DBs, except few other things, concepts & terms. For me it is looking like to solve your issue with different route which may suite your application roadmap.
And I dont have that huge amount data with me to explore these fields properly, but still I will try to put few things during my journey.
In the initial phase I don't see it much different than other DBs, except few other things, concepts & terms. For me it is looking like to solve your issue with different route which may suite your application roadmap.
I am going through-
https://www.tutorialspoint.com/elasticsearch
===============================================================================================
Here I got to know that you can create & decide on the type of data to accept in ElasticSearch like we have in other DBs & yes it helps in long run to maintain the data & search for.
So to study about this mapping concept I read - https://logz.io/blog/elasticsearch-mapping/
And the correct example in tutorialspoint will be like -
PUT http://localhost:9200/bankdetails/
{
"mappings": {
"account": {
"properties": {
"name": {
"type": "text",
"copy_to": "data"
},
"date": {
"type": "date",
"copy_to": "data"
},
"balance": {
"type": "double"
},
"liability": {
"type": "double"
},
"data": {
"type": "text"
}
}
}
}
}
And the information about the above syntax you will get from the second given above.
For "copy_to", it is used to combine the data of multiple attributes & search them as single value. Do search on google for this to understand more. https://www.elastic.co/guide/en/elasticsearch/reference/current/copy-to.html
Here, this mapping concept are bit different in different versions of Elasticsearch.
Below I have just changed the above mapping definition a bit -
{
"mappings": {
"account": {
"properties": {
"date": {
"type": "date"
},
"balance": {
"type": "double"
},
"data": {
"type": "text"
},
"liability": {
"type": "double"
},
"first_name": {
"type": "text",
"copy_to": "data"
},
"last_name": {
"type": "text",
"copy_to": "data"
}
}
}
}
}
So what is 'account' parameter used above, it defines the document type in Elasticsearch & will be used while inserting the data in the index 'bankdetails'. If you try to skip document type parameter then it will complain & will not be able to create the mapping for the index. So mention this with any name of your choice but it is needed here.
Similarly when we insert the data in the above index, we need to mention the document type name used while creating mapping like shown below -
POST http://localhost:9200/bankdetails/account/
Above, we are giving the index name in which to insert the data, then have mentioned the document type i.e. 'account'
Note:- Above we are not giving the id of the document so the request should be 'POST' only & below is the data in body of the request -
{
"first_name": "Sanjay",
"last_name": "Singh",
"date": "2017-07-01",
"balance": "1800000",
"liability": "0.3"
}
Here the id will be generated automatically.
But if we want to mention the id for the data being provided then we can use 'PUT' or 'POST' request with the data in body as shown below -
PUT http://localhost:9200/bankdetails/account/1/
Note:- Here the responsibility of giving the correct id is with the user, as if any data exists for that id then it will be replaced with the current data provided.
{
"first_name": "Shilpi",
"last_name": "Tiwari",
"date": "2017-07-01",
"balance": "18000",
"liability": "0.3"
}
Another document will be created with the above data with id as 1.
Now let us give another data with id from our side & now this request will be POST one.
POST http://localhost:9200/bankdetails/account/
{
"first_name": "Nitin",
"last_name": "Agrawal",
"date": "2019-07-31",
"balance": "17000",
"liability": "0.3"
}
And again this document will be having the id created by Elasticsearch.
Let us query our data -
GET http://localhost:9200/bankdetails/_search?pretty=true
Output will be like shown with both our ID & default ones, with each having type parameter as 'account' which we gave while creating mapping -
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 3,
"max_score" : 1.0,
"hits" : [
{
"_index" : "bankdetails",
"_type" : "account",
"_id" : "ZiFrSGwB6s_qWXS61Pv7",
"_score" : 1.0,
"_source" : {
"first_name" : "Sanjay",
"last_name" : "Singh",
"date" : "2017-07-01",
"balance" : "1800000",
"liability" : "0.3"
}
},
{
"_index" : "bankdetails",
"_type" : "account",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"first_name" : "Shilpi",
"last_name" : "Tiwari",
"date" : "2017-07-01",
"balance" : "18000",
"liability" : "0.3"
}
},
{
"_index" : "bankdetails",
"_type" : "account",
"_id" : "ZSFoSGwB6s_qWXS6yfum",
"_score" : 1.0,
"_source" : {
"first_name" : "Nitin",
"last_name" : "Agrawal",
"date" : "2019-07-31",
"balance" : "17000",
"liability" : "0.3"
}
}
]
}
}
Now we have got the data above & we used 'copy_to' while creating mapping, so let us use that while to query for that data -
Below is the syntax for the POST query for the url where we are also mentioning the type name for the type of data we query.
http://localhost:9200/bankdetails/account/_search/
Below request you will send as part of the body -
{
"query": {
"match": {
"data": {
"query": "Nitin Agrawal",
"operator": "and"
}
}
}
}
Note 'operator' parameter with 'and'. Here it will search for those document with the field value having both the words given in query to give the below output -
{
Now if we change the above query & operator value like -
{
"query": {
"match": {
"data": {
"query": "Nitin Shilpi",
"operator": "or"
}
}
}
}
Now it will present those documents with fields having any of the given 2 words, so the output will be -
{
===============================================================================================
Being on Development machine, you want to start more than one node of ElasticSearch. So you can start the first node liek normal one but for further nodes you need to make chnge in the execution command -
elasticsearch.bat -Epath.data=data1 -Epath.logs=log1
===============================================================================================
https://www.tutorialspoint.com/elasticsearch
===============================================================================================
Here I got to know that you can create & decide on the type of data to accept in ElasticSearch like we have in other DBs & yes it helps in long run to maintain the data & search for.
So to study about this mapping concept I read - https://logz.io/blog/elasticsearch-mapping/
And the correct example in tutorialspoint will be like -
PUT http://localhost:9200/bankdetails/
{
"mappings": {
"account": {
"properties": {
"name": {
"type": "text",
"copy_to": "data"
},
"date": {
"type": "date",
"copy_to": "data"
},
"balance": {
"type": "double"
},
"liability": {
"type": "double"
},
"data": {
"type": "text"
}
}
}
}
}
And the information about the above syntax you will get from the second given above.
For "copy_to", it is used to combine the data of multiple attributes & search them as single value. Do search on google for this to understand more. https://www.elastic.co/guide/en/elasticsearch/reference/current/copy-to.html
Here, this mapping concept are bit different in different versions of Elasticsearch.
Below I have just changed the above mapping definition a bit -
{
"mappings": {
"account": {
"properties": {
"date": {
"type": "date"
},
"balance": {
"type": "double"
},
"data": {
"type": "text"
},
"liability": {
"type": "double"
},
"first_name": {
"type": "text",
"copy_to": "data"
},
"last_name": {
"type": "text",
"copy_to": "data"
}
}
}
}
}
So what is 'account' parameter used above, it defines the document type in Elasticsearch & will be used while inserting the data in the index 'bankdetails'. If you try to skip document type parameter then it will complain & will not be able to create the mapping for the index. So mention this with any name of your choice but it is needed here.
Similarly when we insert the data in the above index, we need to mention the document type name used while creating mapping like shown below -
POST http://localhost:9200/bankdetails/account/
Above, we are giving the index name in which to insert the data, then have mentioned the document type i.e. 'account'
Note:- Above we are not giving the id of the document so the request should be 'POST' only & below is the data in body of the request -
{
"first_name": "Sanjay",
"last_name": "Singh",
"date": "2017-07-01",
"balance": "1800000",
"liability": "0.3"
}
Here the id will be generated automatically.
But if we want to mention the id for the data being provided then we can use 'PUT' or 'POST' request with the data in body as shown below -
PUT http://localhost:9200/bankdetails/account/1/
Note:- Here the responsibility of giving the correct id is with the user, as if any data exists for that id then it will be replaced with the current data provided.
{
"first_name": "Shilpi",
"last_name": "Tiwari",
"date": "2017-07-01",
"balance": "18000",
"liability": "0.3"
}
Another document will be created with the above data with id as 1.
Now let us give another data with id from our side & now this request will be POST one.
POST http://localhost:9200/bankdetails/account/
{
"first_name": "Nitin",
"last_name": "Agrawal",
"date": "2019-07-31",
"balance": "17000",
"liability": "0.3"
}
And again this document will be having the id created by Elasticsearch.
Let us query our data -
GET http://localhost:9200/bankdetails/_search?pretty=true
Output will be like shown with both our ID & default ones, with each having type parameter as 'account' which we gave while creating mapping -
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 3,
"max_score" : 1.0,
"hits" : [
{
"_index" : "bankdetails",
"_type" : "account",
"_id" : "ZiFrSGwB6s_qWXS61Pv7",
"_score" : 1.0,
"_source" : {
"first_name" : "Sanjay",
"last_name" : "Singh",
"date" : "2017-07-01",
"balance" : "1800000",
"liability" : "0.3"
}
},
{
"_index" : "bankdetails",
"_type" : "account",
"_id" : "1",
"_score" : 1.0,
"_source" : {
"first_name" : "Shilpi",
"last_name" : "Tiwari",
"date" : "2017-07-01",
"balance" : "18000",
"liability" : "0.3"
}
},
{
"_index" : "bankdetails",
"_type" : "account",
"_id" : "ZSFoSGwB6s_qWXS6yfum",
"_score" : 1.0,
"_source" : {
"first_name" : "Nitin",
"last_name" : "Agrawal",
"date" : "2019-07-31",
"balance" : "17000",
"liability" : "0.3"
}
}
]
}
}
Now we have got the data above & we used 'copy_to' while creating mapping, so let us use that while to query for that data -
Below is the syntax for the POST query for the url where we are also mentioning the type name for the type of data we query.
http://localhost:9200/bankdetails/account/_search/
Below request you will send as part of the body -
{
"query": {
"match": {
"data": {
"query": "Nitin Agrawal",
"operator": "and"
}
}
}
}
Note 'operator' parameter with 'and'. Here it will search for those document with the field value having both the words given in query to give the below output -
{
- "took": 2,
- "timed_out": false,
- "_shards": {
- "total": 5,
- "successful": 5,
- "skipped": 0,
- "failed": 0
- "hits": {
- "total": 1,
- "max_score": 0.5753642,
- "hits": [
- {
- "_index": "bankdetails",
- "_type": "account",
- "_id": "ZSFoSGwB6s_qWXS6yfum",
- "_score": 0.5753642,
- "_source": {
- "first_name": "Nitin",
- "last_name": "Agrawal",
- "date": "2019-07-31",
- "balance": "17000",
- "liability": "0.3"
- {
Now if we change the above query & operator value like -
{
"query": {
"match": {
"data": {
"query": "Nitin Shilpi",
"operator": "or"
}
}
}
}
Now it will present those documents with fields having any of the given 2 words, so the output will be -
{
- "took": 1,
- "timed_out": false,
- "_shards": {
- "total": 5,
- "successful": 5,
- "skipped": 0,
- "failed": 0
- "hits": {
- "total": 2,
- "max_score": 0.2876821,
- "hits": [
- {
- "_index": "bankdetails",
- "_type": "account",
- "_id": "1",
- "_score": 0.2876821,
- "_source": {
- "first_name": "Shilpi",
- "last_name": "Tiwari",
- "date": "2017-07-01",
- "balance": "18000",
- "liability": "0.3"
- ,
- {
- "_index": "bankdetails",
- "_type": "account",
- "_id": "ZSFoSGwB6s_qWXS6yfum",
- "_score": 0.2876821,
- "_source": {
- "first_name": "Nitin",
- "last_name": "Agrawal",
- "date": "2019-07-31",
- "balance": "17000",
- "liability": "0.3"
- {
===============================================================================================
Being on Development machine, you want to start more than one node of ElasticSearch. So you can start the first node liek normal one but for further nodes you need to make chnge in the execution command -
elasticsearch.bat -Epath.data=data1 -Epath.logs=log1
===============================================================================================