Database « Intelligrape Groovy & Grails Blogs

Archive for the ‘ Database ’ Category

How to Determine Average in MongoDB

Posted by on April 2nd, 2013

MongoDB provides several ways of computing the average value of a group in a collection. One of the simplest ways of determining average is using the method db.collection.group().

The method db.collection.group() bunches the documents of a collection on the basis of the keys mentioned and executes aggregation functions on them. This method can be used for performing several operations, including count and addition. In order words, the method performs functionality similar to that of the SQL’s Select-Group By combination.

The implementation and working of the method db.collection.group() for determining the average of a field in a collection is illustrated here using the example of a collection ‘userquestions’. The query, which can be used to find average, is as follows -


var reduceFunction = function(currentDocument, previousDocument) {
     previousDocument.totalRating += currentDocument.rating; previousDocument.count += 1
};

var finalizeFunction = function(currentDocument) {
     currentDocument.average = currentDocument.totalRating/currentDocument.count;
     delete currentDocument.totalRating;
     delete currentDocument.count;
};

db.userquestions.group({
     key: {'quesId': true},
     initial: {totalRating: 0, count:0},
     reduce: reduceFunction,
     finalize:finalizeFunction
});

In the present context, the value of the ‘key’ is the field ‘quesId’. Documents are grouped together on the basis of this key. The initial values for the parameters ‘count’ and ‘totalRating’ are set to zero. Besides these, two functions are declared and defined in the query. The function reduce totals the value of the ‘rating’ and saves it in ‘totalRating’. In addition to this, it also maintains the count of documents processed.

On the other hand, the function ‘finalize’ determines the average value of rating by dividing the value of ‘totalRating’ by the count.The execution of this query returns an array containing different ‘quesId’, which represents the different groups and the corresponding average rating for each of these groups.

Hope this will help. :-)

Thanks,
Sakshi Tyagi
sakshi@intelligrape.com

Posted in Database

Copying Redis Database from one server to another

Posted by on February 20th, 2013

In my recent project, I needed to copy the redis database from one server to another. Although redis provides SAVE and BGSAVE commands, I found another cool way of doing it.

Log in to the new server and execute the following command in terminal:

redis-cli slaveof IP-ADDRESS-OF-OLD-SERVER 6379

The above command starts replicating the data stored in redis database of old server to redis database of new server. Here, 6379 is the default port which redis uses. Once the replication is done, we issue the following command :

redis-cli slaveof no one

Now we have the entire redis database copied to the new server.

Raj Gupta
raj.gupta@intelligrape.com
@rajdgreat007

Posted in Database, Linux

Create game Leaderboard using sqlProjection

Posted by on November 8th, 2012

Recently I was working on a Facebook game application using Grails 2.0 and Postgres database. The use case was to show the game leaderboard with ranks of the players. Players with same score should be given same rank and next player rank should be incremented rank of last player. Here is the sample data for players and expected result:
game score

There is a rank() function in Postgres which solved my problem. Here is the sample query :

select rank() over(order by score desc) as rnk, name, score from player

The good news is that Grails 2.2 supports sqlProjection where you can use native sql projections like this :

Player.createCriteria().list{
  projections {
      sqlProjection ('rank() over(order by score desc) as rnk',
                     ['rnk'], [org.hibernate.Hibernate.INTEGER])
       property('name')
       property('score')
    }
  }
.each {  println "${it[0]} \t ${it[1]} \t ${it[2]}"}

If you are using older version of Grails then you can go with following alternatives:

  • Using sessionFactory bean and createSQLQuery
  • String query="select rank() over(order by score desc) as rnk, name, score from player"
    //inject sessionFactory bean object (def sessionFactory)
    sessionFactory.currentSession.createSQLQuery(query).list()
    .each {  println "${it[0]} \t ${it[1]} \t ${it[2]}"}
    
  • Using hibernate criteria query
  • /* import org.hibernate.Hibernate
    import org.hibernate.type.Type
    import org.hibernate.criterion.Projections
    import org.hibernate.criterion.Restrictions */
    
    sessionFactory.currentSession.createCriteria(Player)
                    .setProjection(Projections.projectionList()
                    .add(Projections.sqlProjection(
                                     "rank() over(order by score desc) as rnk",
                                     ["rnk"] as String[], 
                                     [Hibernate.INTEGER] as Type[]))
                    .add(Projections.property("name"))
                    .add(Projections.property("score")))
             .list()
             .each {println "${it[0]} \t ${it[1]} \t ${it[2]}"}
    

I have not been able to find a solution to find rank of specific user without loading all the records/rows and doing a groovy find on the result. The rank function assigns rank to each row based on the final result set returned by the query. So if you add a restriction to select only specific player’s rank, it will be always 1 (as the final result set will have only one record).

Hope you will share a better way to find rank of specific user :)

Cheers!
~~Bhagwat Kumar~~
bhagwat(at)intelligrape(dot)com
http://twitter.com/bhagwatkumar
http://in.linkedin.com/in/bhagwatkumar

Posted in Database, GORM, Grails

Accessing remote data through cross-domain ajax call in jquery

Posted by on September 24th, 2012

While developing a mobile app using phonegap ( or otherwise also :) ), we can access remotely hosted mysql database using jquery ajax calls. But this interaction between jquery and mysql database cannot happen directly. We will need to specify a server side script(in PHP terminology) or a controller action(in Grails Terminology) that will fetch data from the mysql database and serve it to the jquery call. Jquery will simply make a cross-domain ajax request to the server side script and the script will send requested data as response.

For a successful cross-domain communication, we need to use dataType “jsonp” in jquery ajax call.
JSONP or “JSON with padding” is a complement to the base JSON data format which provides a method to request data from a server in a different domain, something prohibited by typical web browsers.

When we specify dataType as jsonp, a “callback” parameter is appended to the request url and jquery creates a function whose name is the value of callback parameter. On server side, the script receives the “callback” parameter value(which is name of the function) and sends the data as argument to that function. Alternatively, that data is also available in the success function of jquery.

Jquery Code :

	    function crossDomainCall(url,data,fnSuccess,fnError){
            $.ajax({
                type:'POST',
                url:url,
                contentType:"application/json",
                dataType:'jsonp',
                crossDomain:true,
                data:data,
                success:fnSuccess,
                error: fnError
            });
        }
	
        function authenticateUser(username, password) {
            var url = 'http://www.example.com/user/authenticate';
            var data={username:username, password:password};
            var fnSuccess=function (dataReceived) {
		    if(dataReceived) {
			    alert("Welcome "+dataReceived.name);
		    }else{
			    alert("Authentication failed")
		    }
            };

            var fnError=function (e) {
                alert(e);
            };
            crossDomainCall(url,data,fnSuccess,fnError);
        }

Server side code :

 
	def authenticate(String username, String password){
        	User user=User.findByNameAndPassword(username,password)
		    if (user) {
		        render "${params.callback}(${user as JSON})"
		    }else{
		        render "${params.callback}(null)"
		    }
    	}

Here, the function name is received from params.callback and data is sent in json form as an argument to the function.

Hope it helps.
Have Fun :)

Extract correct class from hibernate object wrapped with javassist

Posted by on September 21st, 2012

Hi,

Some times when we access domain objects of collections in one to many or many to many relationships, we get Hibernate class wrapped with javassist, basically hibernate wraps all loaded objects with javassist after loading(lazy loading case) them .
(more…)

Posted in Database, GORM, Grails

Logging to remote MySQL Server using SSH tunneling

Posted by on September 17th, 2012

Very often, we all need to access a remote database for debugging or any other related stuff.

The simplest thing that comes to mind is to take the dump of remote database and bring it to the local and run the application using that data.

We can avoid that, if we can connect our local mysql client to a remote machine over a SSH connection so that we can run queries on remote machine from our client machine, without dumping the database of remote machine.

To do that, we need to just need to remote machine over SSH with -L option


ssh -L 32001:127.0.0.1:3306 ubuntu@hostname.com

This means that we redirected call on port number 32001 to port number 3306 on the remote machine and all responses from the remote machine on 3306 will be redirected to 32001 on the local machine.

All we need to do now is to open mysql client and attach it to port number 32001 on local machine, we need to do


mysql -h 127.0.0.1 -P 32001 -u'username' -p'password'

Here we are connecting to local machine on port number 32001, with username and password for that of the remote machine.

Thanks

Sachin Anand

sachin@intelligrape.com

@babasachinanand

Posted in Database, Linux

Mapping duplicate domain class name having different packages using autoImport in Grails

Posted by on September 14th, 2012

Hi,

Recently one of my friend asked me whether there is a mechanism in grails to handle the use-case where we have different packages but having same domain name. Here we are having duplicate domain class names in different package and by default the domain classes are auto-imported in HQL queries which doesn’t required to specify whole class name. Hence due to it class name are no longer unique because of this it cause DuplicateMappingException in our case. For this I refer my friend to set autoImport false in mapping.

Grails provide us a mapping key named ‘autoImport‘. We can disable the auto-import for any one of the domain or both which will fix our problem to some extent. Lets see how it helps :

Suppose we have 2 packages named :
1. com.tarun.poc.project having Domain named “Issue”.

package com.tarun.poc.project
class Issue {
    String summary
    static constraints = {
    }
}

2. com.tarun.poc.company having Domain named “Issue”.

package com.tarun.poc.company
class Issue {
    String description
    static constraints = {
    }
}

When you run your app you will recieve the exception some thing like this :

nested exception is org.hibernate.DuplicateMappingException: duplicate import: Issue refers to both com.tarun.poc.project.Issue and com.tarun.poc.company.Issue

Now we add autoImport to the mapping of domain ‘Issue’ one or both and set it to false as specified below :

package com.tarun.poc.project
class Issue {
    String summary
    static constraints = {
    }
    static mapping = {
        autoImport false
        table 'project_issue'
    }
}

It will fix your problem regarding ‘DuplicateMappingException’. If you see above we have also specified table mapping because if we didn’t specify table mapping it will create a single table in database with name ‘issue’ and having properties of both the domains into that table.

Auto Import without Table Mapping

Hence to create different tables for Domain Class or customizes the name of the database table associated with the domain class we used table mapping.

Image of Auto Import with Table Mapping

Hope it helps :)

Thanks,
Tarun Pareek
tarun@intelligrape.com
http://in.linkedin.com/in/tarunpareek
https://twitter.com/TarunPareek_
More Blogs by Me

Posted in Database, Grails, Groovy

Toggling foreign key checks in MySQL

Posted by on September 4th, 2012

Recently in my project, we needed to import database dumps of a legacy database in MySQL, which was involved in a nested relationship where one tuple could be the parent record of another tuple. We created the appropriate domain structure and checked that the foreign key references were created accordingly. All well and good.

So we started to restore the dump for the database. But soon problems started arising when certain records had references to records that were not yet created i.e, that is some records were being assigned parents that did not exist. We needed a way to turn of foreign key checking for some time. So after taking a look at the MySQL documentation (http://dev.mysql.com/doc/refman/5.1/en/innodb-foreign-key-constraints.html), I stumbled across a way to turn off foreign key checking by running the following command:

mysql> SET FOREIGN_KEY_CHECKS = 0
mysql> SOURCE fakeDB
mysql> SET FOREIGN_KEY_CHECKS = 1

So there it was. We had happily imported our legacy database with all the foreign key checks intact.

If you’re not sure about the status of foreign key checking in your MySQL instance, you can use the following command to check the status:

mysql> select * from INFORMATION_SCHEMA.GLOBAL_VARIABLES where VARIABLE_NAME like 'FOREIGN_KEY_CHECKS';

Hope this helps.

Cheers
Roni C Thomas
roni[at]intelligrape[dot]com
@ronicthomas

Posted in Database

Optimistic locking strategy in Grails

Posted by on September 1st, 2012

In Grails optimistic locking is achieved by version property . The Grails Domain classes has a built in property called “version”. This property can be used for optimistic locking. Although you can remove this property

static mapping ={
      version false
}

but its not a very good practice. The initial value of version field is 0 and Grails automatically increments it by 1 every time the object is updated. This property helps to identify whether the currently updating object has already been updated or not.

 

Suppose we have a Book domain and its corresponding BookController the default Grails scaffold update action has code something like as follows

if (version != null) {
            if (book.version > version) {
                book.errors.rejectValue("version", "default.optimistic.locking.failure",
                        [message(code: 'book.label', default: 'Book')] as Object[],
                        "Another user has updated this Book while you were editing")
                render(view: "edit", model: [book: book])
                return
            }
        }

As we can see it checks the current version of the object with the version got from request. If the current version of object is greater, it injects the error into the object and object is not saved.

 

Hope it helps
Uday Pratap Singh
uday@intelligrape.com
https://twitter.com/meudaypratap
http://in.linkedin.com/in/meudaypratap

Posted in Database, GORM, Grails

Using Apache to save data in redis

Posted by on July 30th, 2012

In one of my projects, I was using redis database to collect some statistics and I thought of saving data into it at apache level. This would considerably enhance the speed of saving data as it would not require the interception of grails to save data.

The first step for this was to install apache by firing the following command in terminal :

sudo apt-get install apache2

After installing apache, it was required to set up a site. For that I created a file named www.raj.com in /etc/apache2/sites-available directory and configured it as follows:

<VirtualHost *:80>
ServerAdmin webmaster@localhost
ServerName www.raj.com
ServerAlias raj.com

DocumentRoot /usr/lib/cgi-bin

<Directory />
Options FollowSymLinks
AllowOverride All
Order allow,deny
</Directory>

ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory "/usr/lib/cgi-bin">
AllowOverride All
AddHandler cgi-script .cgi
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>

ErrorLog ${APACHE_LOG_DIR}/error.log

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn

CustomLog ${APACHE_LOG_DIR}/access.log combined

Alias /doc/ "/usr/share/doc/"
<Directory "/usr/share/doc/">
Options Indexes MultiViews FollowSymLinks
AllowOverride All
Order deny,allow
Deny from all
Allow from 127.0.0.0/255.0.0.0 ::1/128
</Directory>

</VirtualHost>

Here the ServerName is set to www.raj.com and DocumentRoot is set to /usr/lib/cgi-bin. It means that if I hit url www.raj.com, then i will see the contents of /usr/lib/cgi-bin directory (This will come into action after enabling the site).
AddHandler cgi-script .cgi means that we are adding a cgi-script handler for files whose extension is cgi.

Now my site is in the list of available sites, but it’s still not enabled. To enable it I fired the following command in terminal :

a2ensite www.raj.com

This command created a soft link of my site into /etc/apache2/sites-enabled directory and now my site was enabled.

The last step was to make an entry for my site in  /etc/hosts. I appended following line to /etc/hosts to do so :

127.0.0.1 www.raj.com

Finally, i was required to reload apache using command

service apache2 reload 

And now my site was live. To save data into the redis database, i created a file named saveData.cgi in /usr/lib/cgi-bin folder, given execution permissions to it and entered the following script into it.


#!/bin/sh
echo "Content-type: text/html\n\n"

date="na"
partnerSite="na"
videoId="na"

date=`echo $QUERY_STRING | sed 's/.*date\=\([^&]\+\).*/\1/'`

partnerSite=`echo $QUERY_STRING | sed 's/.*partnerSite\=\([^&]\+\).*/\1/'`

videoId=`echo $QUERY_STRING | sed 's/.*videoId\=\([^&]\+\).*/\1/'`

`redis-cli hincrby "$videoId-$date-views" $partnerSite 1`

Here shebang is used to execute script in the bash shell and the content type is set to text/html.
In redis, HINCRBY command is used in relation with hashes, where a key has one or more fields each with a corresponding value.
If the key and field already exists in the redis database, HINCRBY command would increment the value of the field by specified number. If the key doesn’t exists, HINCRBY creates a key with specified filed and value.

Thus, if i hit the url http://www.raj.com/saveData.cgi?date=20July2012&partnerSite=facebook&videoId=wwYXWU in my browser, then date, partnerSite and videoId are extracted from the query string using sed command and in the redis database an entry would be saved whose key will be “wwYXWU-20July2012-views”, field will be “facebook” and value will be 1. If the same url is received again, the value of field “facebook” of key “wwYXWU-20July2012-views” will become 2 and so on.

In this way, we can easily use apache to save and update data in redis database.

Posted in Database, Grails, Linux