MySql « Intelligrape Groovy & Grails Blogs

Posts Tagged ‘ MySql ’

Accessing remote data through cross-domain ajax call in jquery

Posted by on September 24th, 2012

While developing a mobile app using phonegap ( or otherwise also :) ), we can access remotely hosted mysql database using jquery ajax calls. But this interaction between jquery and mysql database cannot happen directly. We will need to specify a server side script(in PHP terminology) or a controller action(in Grails Terminology) that will fetch data from the mysql database and serve it to the jquery call. Jquery will simply make a cross-domain ajax request to the server side script and the script will send requested data as response.

For a successful cross-domain communication, we need to use dataType “jsonp” in jquery ajax call.
JSONP or “JSON with padding” is a complement to the base JSON data format which provides a method to request data from a server in a different domain, something prohibited by typical web browsers.

When we specify dataType as jsonp, a “callback” parameter is appended to the request url and jquery creates a function whose name is the value of callback parameter. On server side, the script receives the “callback” parameter value(which is name of the function) and sends the data as argument to that function. Alternatively, that data is also available in the success function of jquery.

Jquery Code :

	    function crossDomainCall(url,data,fnSuccess,fnError){
            $.ajax({
                type:'POST',
                url:url,
                contentType:"application/json",
                dataType:'jsonp',
                crossDomain:true,
                data:data,
                success:fnSuccess,
                error: fnError
            });
        }
	
        function authenticateUser(username, password) {
            var url = 'http://www.example.com/user/authenticate';
            var data={username:username, password:password};
            var fnSuccess=function (dataReceived) {
		    if(dataReceived) {
			    alert("Welcome "+dataReceived.name);
		    }else{
			    alert("Authentication failed")
		    }
            };

            var fnError=function (e) {
                alert(e);
            };
            crossDomainCall(url,data,fnSuccess,fnError);
        }

Server side code :

 
	def authenticate(String username, String password){
        	User user=User.findByNameAndPassword(username,password)
		    if (user) {
		        render "${params.callback}(${user as JSON})"
		    }else{
		        render "${params.callback}(null)"
		    }
    	}

Here, the function name is received from params.callback and data is sent in json form as an argument to the function.

Hope it helps.
Have Fun :)

Logging to remote MySQL Server using SSH tunneling

Posted by on September 17th, 2012

Very often, we all need to access a remote database for debugging or any other related stuff.

The simplest thing that comes to mind is to take the dump of remote database and bring it to the local and run the application using that data.

We can avoid that, if we can connect our local mysql client to a remote machine over a SSH connection so that we can run queries on remote machine from our client machine, without dumping the database of remote machine.

To do that, we need to just need to remote machine over SSH with -L option


ssh -L 32001:127.0.0.1:3306 ubuntu@hostname.com

This means that we redirected call on port number 32001 to port number 3306 on the remote machine and all responses from the remote machine on 3306 will be redirected to 32001 on the local machine.

All we need to do now is to open mysql client and attach it to port number 32001 on local machine, we need to do


mysql -h 127.0.0.1 -P 32001 -u'username' -p'password'

Here we are connecting to local machine on port number 32001, with username and password for that of the remote machine.

Thanks

Sachin Anand

sachin@intelligrape.com

@babasachinanand

Posted in Database, Linux

Toggling foreign key checks in MySQL

Posted by on September 4th, 2012

Recently in my project, we needed to import database dumps of a legacy database in MySQL, which was involved in a nested relationship where one tuple could be the parent record of another tuple. We created the appropriate domain structure and checked that the foreign key references were created accordingly. All well and good.

So we started to restore the dump for the database. But soon problems started arising when certain records had references to records that were not yet created i.e, that is some records were being assigned parents that did not exist. We needed a way to turn of foreign key checking for some time. So after taking a look at the MySQL documentation (http://dev.mysql.com/doc/refman/5.1/en/innodb-foreign-key-constraints.html), I stumbled across a way to turn off foreign key checking by running the following command:

mysql> SET FOREIGN_KEY_CHECKS = 0
mysql> SOURCE fakeDB
mysql> SET FOREIGN_KEY_CHECKS = 1

So there it was. We had happily imported our legacy database with all the foreign key checks intact.

If you’re not sure about the status of foreign key checking in your MySQL instance, you can use the following command to check the status:

mysql> select * from INFORMATION_SCHEMA.GLOBAL_VARIABLES where VARIABLE_NAME like 'FOREIGN_KEY_CHECKS';

Hope this helps.

Cheers
Roni C Thomas
roni[at]intelligrape[dot]com
@ronicthomas

Posted in Database

Removing/replacing special character from database.

Posted by on June 11th, 2012

Recently in one of my project i faced a problem that database was having special character, which is shown as space on the User Interface.

So as to solve this issue i found a very simple solution which consists of following steps:

1. First, you just need to identify which type special character to be removed/replaced like in my case it was shown as space but stored as <?> in the db.

For that you can use the query.

select HEX(email) from person;

and you can also refer to this link for ASCII code of character. In my case it was 160.

2.  Now, next step is simple just run this query.

 update person set email=replace(email,char(160),''); 

Thats it !!!  :)

It helps me a lot!!! Hope that helps you guys!!!

Anuj Aneja

Intelligrape Software Pvt. Ltd.

Tags: ,
Posted in Database

Clear Time function for MySQL

Posted by on April 12th, 2012

Hi all,

Here is a simple function that can allow you to clear time from the DATETIME field of your database table

DROP FUNCTION IF EXISTS cleartime;
delimiter //
CREATE FUNCTION cleartime(dt datetime) RETURNS DATETIME
  NO SQL
  DETERMINISTIC
BEGIN
  DECLARE t    varchar(15);               -- the time part of the dt
  DECLARE rdt  datetime default dt;    	  -- the datetime after time part is cleared

  SET t = TIME(dt); 
  SET rdt = SUBTIME(dt,t);

  RETURN rdt;
END //
delimiter ;

To import this function to your database just copy and paste above code in MySQL command prompt with your database selected.

The implementation is shown in following example:

select cleartime(datetime_field) from table_name;     -- to view field with its time cleared
                     --OR
update table_name set datetime_field=cleartime(datetime_feild) from table_name;
Posted in Database

mysqldump of particular records without create – drop of tables

Posted by on March 27th, 2012

Hi,
 
Recently i was in a situation where i need to take the dump of particular records in mysql. We are pretty much familiar with the normal mysqldump statement given below :

mysqldump --user=usr --password=pwd database_name table_name > dumpFile.sql;

The above statement will provide us the dump of the respective database and table into the dumpFile.sql. But if you open the dump file you will see something like this :
 

    //dumpFile.sql
DROP TABLE IF EXISTS `table_name`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `table_name` (
  ...
) ENGINE=InnoDB AUTO_INCREMENT=204182 DEFAULT CHARSET=latin1;

--
-- Dumping data for table `table_name`

LOCK TABLES `table_name` WRITE;
/*!40000 ALTER TABLE `table_name` DISABLE KEYS */;
INSERT INTO `table_name` VALUES ('1','data1');
/*!40000 ALTER TABLE `table_name` ENABLE KEYS */;
UNLOCK TABLES;

Because of the above dump, it will always first drop and recreate the table before sourcing your dump into the database.
So our first task to remove the creation and drop of table from the mysqldump, because when we take dump of particular records, What we want is just inserting the records in database without create and drop of tables and no loss of existing data.
 

    //Creating Dump without create and drop of table :
mysqldump --user=usr --password=pwd --no-create-info database_name table_name > dumpFile.sql;

If you compare the first and above statement there is only one change, –no-create-info option :
It don’t write CREATE TABLE statements that re-create each dumped table. So our dumpFile.sql now contains only this :

--
-- Dumping data for table `table_name`

LOCK TABLES `table_name` WRITE;
/*!40000 ALTER TABLE `table_name` DISABLE KEYS */;
INSERT INTO `table_name` VALUES ('1','data1');
/*!40000 ALTER TABLE `table_name` ENABLE KEYS */;
UNLOCK TABLES;

 

    //Creating Dump of particular records using conditional clause and without create and drop of table :
mysqldump --user=usr --password=pwd --no-create-info database_name table_name --where="id in (1,2,3)"> dumpFile.sql;

In above statement with –no-create-info, we removed the create-drop of table and by –where option we can specify our condition on basis of which we want to fetch the particular records. In above statement it will fetch the record in table that has value 1, 2 or 3 in field ‘id’. So our dumpFile.sql now contains only this :

--
-- Dumping data for table `table_name`
--
-- WHERE:  id in (1,2,3)

LOCK TABLES `table_name` WRITE;
/*!40000 ALTER TABLE `table_name` DISABLE KEYS */;
INSERT INTO `table_name` VALUES ('1','data1'),('2','data2'),('3','data3');
/*!40000 ALTER TABLE `table_name` ENABLE KEYS */;
UNLOCK TABLES;

 
I hope it helps. :)
 
Thanks,
Tarun Pareek
tarun@intelligrape.com
http://in.linkedin.com/in/tarunpareek
More Blogs by Me

Posted in Database

Alphanumeric Sorting using Criteria Query (with MySQL database)

Posted by on February 2nd, 2012

I am working on a Grails application with MySQL database. I had a use case in which I had to implement alphanumeric sorting using Criteria Query on Grails. By alphanumeric sorting I mean if there is a class Employee with field empId then on doing :

 
Employee e1 = new Employee(empId:'emp10').save()
Employee e2 = new Employee(empId:'emp2').save()
Employee e3 = new Employee(empId:'emp1').save()
Employee e4 = new Employee(empId:'1emp').save()
Employee e5 = new Employee(empId:'10emp').save()
Employee e6 = new Employee(empId:'2emp').save()

Employee.createCriteria().list([sort:'empId'])
            /* OR */
Employee.createCriteria().list{
    order('empId')
}

should give result like [e4,e6,e5,e3,e2,e1]. But according to sql sorting the result would be [e5,e4,e6,e3,e1,e2]

One thing to note was that criteria query implemented sorting at the database end.
After googling around, I came across a script that had a SQL function which converted the field value to string that can alphanumerically sorted easily at database end.

To migrate the functions in script to database you have to unzip the script and write following command on terminal:

mysql -u username -p database_name < /path/to/unzipped/directory/natsort.install.mysql

Now only thing I had to do was to get a derived field from the Grails property that had to be sorted alphanumerically which was easily possible as the Grails had ability to create transient field using formula (thanks to wonderful Blog by Uday). Thus I had to create a new transient field using SQL function in formula. So I did something like this :

class Employee {
    String empId
    String empIdSortField           //transient field
    static mapping = {
        empIdSortField formula: 'natsort_canon(empId, "natural")'
    }
}

Now if I had to get Employee objects are sorted by empId alphanumerically, what I would do is :

Employee.createCriteria().list {
    order('empIdSortField')
}

Hope this was helpful to you!

Posted in Database, GORM, Grails

Mysql top just like we have top command in Linux

Posted by on October 8th, 2011
Once in my grails project i was facing the problem of freeze of server.When this happens tomcat becomes unresponsive.Later on by analyzing the thread dump we found that this problem might be due to the database and tomcat communication. So as to analyse how many connection are active at mysql end we found a query.

Show processlist;

Which gives us the result as:
Output:
+—–+——+———–+———————+———+——+——-+——————+

| Id  | User | Host      | db                      | Command | Time | State | Info             |

+—–+——+———–+———————+———+——+——-+——————+

| 242 | root | localhost |      test_db    | Query   |    0 | NULL  | show processlist |

+—–+——+———–+———————+———+——+——-+——————+

The output shows how many connections are active and their properties.
But for this you need to run the query again and again to analyse the connections information. For a better experience we can use mytop  which is very easy to install and just like the top in linux, simple steps need to be followed to make it work.

Step 1:Install mytop using the following command.

anuj@intelligrape:~$ sudo apt-get install mytop
Step 2:
After installation we need make a configuration file .mysqlconfig any where
e.g
anuj@intelligrape:~$ vim .mysqlconfig
user=

pass=

host=localhost

db=db_name

delay=5

port=330

6socket=

batchmode=0

header=1

color=1

idle=1

For more option you can refer to documentation of mytop .
Step 3:
Now just need run this command to see the mysqltop working.
anuj@intelligrape:~$ mytop ~/.mysqlconfig
output:

Thats it ! Internally it runs the query “show processlist” and is very elegant for analysing mysql connections.

This helps me! Hope this helps you guys!
Anuj Aneja

Intelligrape Software

Tags: ,
Posted in Grails

A use case of Bitwise AND

Posted by on April 15th, 2011

Recently, I used bitwise Anding in my grails project. I am sharing the approach that we followed by means of an example.


Suppose we have a domain class Person which can have multiple attributes like Smart, Intelligent, Talkative etc.

What sort of relationship comes to our mind when we see this?
I believe the obvious answer would be Person hasMany Attributes

Next, what if we want to optimize our design so that the searches on the Person becomes better.

Well, this was the question we were facing in our recent project where we finally used the bitwise AND.
We decided to have one field of type Long in our Person class. We named this as attributes . Soon, I ll be explaining why a long for Attribute…

We ensured that the Attributes are never going to be more than 20-30 . So, it became a nice candidate which could utilize the benefit of Bitwise ANDing.

We associated a binary weight to every instance of Attribute domain class.

The rows in the attribute table of our database looked somewhat like this

id | attribute      | weight
1 | Smart          | 1
2 | Talkative      | 2
3 | Intelligent     | 4
4 | Cooperative  | 8
5 | Ignorant       | 16

So, in order to save a Person who is Smart and Cooperative , we would set his attributes field.
weight of Smart + weight of Cooperative.
ie 1+8 = 9.

Suppose our Person table has some entries as follows:

id  | name  | attributes
1   | Tom   |  9
2   | Fred   |  12
3   | John   | 18

If we want to search a person who is Intelligent we need to have a final query like this

Select * from person where attributes&4 = 4
(4 is the weight of Attribute - Intelligent)

Search a person who is Cooperative and Talkative

Select * from person where attributes&2=2 and attributes&8=8
or more optimally,
Select * from person where attributes&10=10
(2 is the weight of Talkative and 8 is the weight of Cooperative)


Some points to be kept in mind before taking this approach:

There is a maximum limit of 64 values for the field on which we plan to do BitwiseANDing.
Bitwise AND is not directly supported by Hibernate, but we can add a custom function and dialect.



Hope this helped.

Thanks & Regards
Mohd Farid

Posted in Database, Grails

Mysql Dump multipurpose script.

Posted by on January 12th, 2011

Taking MYSQL dump is usual requirement in my project . And Since some of the tables in project are huge so often I have to take dumps of selected tables and often have to ignore some tables.

since we had to do it frequently in our project so I created a script  which does following for me.

  1. Takes database dump
  2. Takes dump of specified tables.
  3. Takes dump of all the tables other than specified.

Inside the script I have added variables like username and password which are required by Mysql. (So if you are using this ,you will need to change them.)

USER="username"
PASSWORD="password"
DATABASE="$1"
DUMPS="Dumps"
DUMPDIR="$DATABASE$DUMPS"
FILENAME="latest.sql"
cd;mkdir "$DUMPDIR"

if [ $# -eq 0 ]; then
	echo "Usage : <databasename> <tablename*>  # if table name(s) is specified only that/those table will be dumped"
	echo "Altername Usage : <databasename> -k <table(s)dToBeIgnored>"
	exit
fi

if [ $# -eq 1 ];then
echo "Dumping $DATABASE to  $DUMPDIR/$FILENAME"
mysqldump --user="$USER" --password="$PASSWORD" "$DATABASE" > ~/"$DUMPDIR"/"$FILENAME"
fi

COMMAND="mysqldump --user=$USER --password=$PASSWORD $DATABASE"
if [ $# -gt 1 -a $2 != "-i" ];
	then
     for i in $*
	 do
		 if [ $i != "$DATABASE" -a $i != "-i" ];then
			 $COMMAND $i > ~/"$DUMPDIR"/"$i.sql";
		 fi
	 done
fi

if [ $# -gt 2 -a $2 == "-i" ];
	then
	COMMAND2=" "
	for i in $*
	do
		if [ $i != "$DATABASE" -a $i != "-i" ];then
		COMMAND2="$COMMAND2 --ignore-table=$DATABASE.$i"
		fi
	done
	$COMMAND $COMMAND2 >  ~/"$DUMPDIR"/"$DATABASE"-"Ignored.sql"
fi

Now what this can be used in three ways -
(Note – I have named this script sqlDump and all of following examples use this name )

  1. ./sqlDump.sh databasename
    This will take a complete dump of all my tables in specified database and store it in file named latest.sql
  2. ./sqlDump.sh databasename table1 table2 … tablenN
    This will take dump of all tables specified.And will store them in file named – table1.sql , table2.sql … tableN.sql
  3. ./sqlDump.sh databasename -i table1 table2 .. tableN
    This will take dump off all the tables in database ignoring specified tables. And will store it in file named <databaseName>-Ignored.sql.

If file is executed without any arguments it will just describe its Usage
Note – All of these tables will be stored in directory named <databasename>Dumps in home directory.

(more…)

Posted in Grails, Linux