Database « Intelligrape Groovy & Grails Blogs

Posts Tagged ‘ Database ’

PostgreSQL with Grails

Posted by on September 5th, 2012

Currently in my Grails project I am using PostgreSQL database so I thought to share my knowledge with everyone. I am using it on Linux operating system. I am mentioning all the steps that I followed to integrate PostgreSQL with Grails .


Step 1:
Install PostgreSQL on your system
To install postgreSql

sudo apt-get install postgresql 

By default the username is postgres

For changing password:

sudo passwd postgres

It will ask for new password. Give any password eg: newPassword

su postgres    //switch to the user postgres with password newPassword
psql


Step 2: Download PostgreSQL

Download latest PostgreSQL JDBC driver and copy it into your application’s lib folder. For example: myProject/lib/postgresql9.1.216.jdbc3.jar


Step 3: Change DataSource file settings

dataSource {
    pooled = true
    driverClassName = "org.postgresql.Driver"
    username = "postgres"
    password = "newPassword"
    dialect = org.hibernate.dialect.PostgreSQLDialect

}
hibernate {
    cache.use_second_level_cache = true
    cache.use_query_cache = true
    cache.provider_class='org.hibernate.cache.EhCacheProvider'
}

environments {
    development {
        dataSource {
            url="jdbc:postgresql://localhost:5432/appName"
            dbCreate = "create-drop"
            driverClassName = "org.postgresql.Driver"
            username = "postgres"
            password = "newPassword"
        }
    }
}

Step 4: Create database

sudo -u postgres createdb <databaseName>

eg: sudo -u postgres createdb myDB

su postgres   //switch to the user postgres with password newPassword

Type \l to show all databases in postgreSql

postgres=# \l

psql -U postgres -d myDB    // now you are connected with your application
\d                         // list all the tables in that database 

View details of a table structure:

\d tablename

eg: postgres=# \d customer

Some useful postgreSQL commands:
Create database

sudo -u postgres createdb <databaseName>   

Drop database

dropdb myDB       

Access database

psql mydb         

Show databases

postgres=#\l             

Get help

postgres=#\h           

Dump database

pg_dump myDB > db.out   

Reload database

psql -d database -f db.out 


Disconnect from psql:

postgres=#\q 


Note:
In my project I got problem while using “user” table name because user is a keyword in PostgreSQL so I changed the name of user table. Instead of changing domain name I just mentioned the table name corresponding to User domain in mapping closure.

Also, one should keep in mind that postgresql maintains id in sequence unlike mysql. It means that the id generator generates id in continous sequence for all the tables instead of generating from 1 for each table.

Hope this will help you. :)
Gunpreet Bedi
gunpreet@intelligrape.com
http://in.linkedin.com/in/gunpreet89
https://twitter.com/gunpreet_ginny

Posted in Grails

Removing/replacing special character from database.

Posted by on June 11th, 2012

Recently in one of my project i faced a problem that database was having special character, which is shown as space on the User Interface.

So as to solve this issue i found a very simple solution which consists of following steps:

1. First, you just need to identify which type special character to be removed/replaced like in my case it was shown as space but stored as <?> in the db.

For that you can use the query.

select HEX(email) from person;

and you can also refer to this link for ASCII code of character. In my case it was 160.

2.  Now, next step is simple just run this query.

 update person set email=replace(email,char(160),''); 

Thats it !!!  :)

It helps me a lot!!! Hope that helps you guys!!!

Anuj Aneja

Intelligrape Software Pvt. Ltd.

Tags: ,
Posted in Database

Handling corrupted references through ignoreNotFound database mapping in Grails

Posted by on April 17th, 2012

Hi,
 
Recently i had a use-case where we have the legacy database and it contains the corrupted references of a non existent record in many-to-one relationship, and i have to populate a grid that contain info also from referenced record. As referenced record doesn’t exist, So when we refer to the certain record will result in Hibernate throwing exception :

org.hibernate.ObjectNotFoundException

and not a single record loaded on grid.
 
As it is legacy database i am not sure what should be the correct value for the particular record.
 
After few searches, i come accross a property in grails database mapping to handle such cases, let see how it work :

class Task{
   String name
   ParentTask parent

   static mapping = {
       parent ignoreNotFound: true
   }
}

class ParentTask{
   String name
}

Basically, ignoreNotFound simply mapping hibernate not-found property.
 
Note : A record loaded with ignoreNotFound: true will throw an exception during call of save() because of the missing reference.
 
Either you can first update the invalid property or corrupted reference to the correct reference and then update other property that will solve your problem. It depend how you handle such scenario on your usecase.
 
Thanks,
Tarun Pareek
tarun@intelligrape.com
http://in.linkedin.com/in/tarunpareek
More Blogs by Me
 

Posted in Database, Grails

Clear Time function for MySQL

Posted by on April 12th, 2012

Hi all,

Here is a simple function that can allow you to clear time from the DATETIME field of your database table

DROP FUNCTION IF EXISTS cleartime;
delimiter //
CREATE FUNCTION cleartime(dt datetime) RETURNS DATETIME
  NO SQL
  DETERMINISTIC
BEGIN
  DECLARE t    varchar(15);               -- the time part of the dt
  DECLARE rdt  datetime default dt;    	  -- the datetime after time part is cleared

  SET t = TIME(dt); 
  SET rdt = SUBTIME(dt,t);

  RETURN rdt;
END //
delimiter ;

To import this function to your database just copy and paste above code in MySQL command prompt with your database selected.

The implementation is shown in following example:

select cleartime(datetime_field) from table_name;     -- to view field with its time cleared
                     --OR
update table_name set datetime_field=cleartime(datetime_feild) from table_name;
Posted in Database

mysqldump of particular records without create – drop of tables

Posted by on March 27th, 2012

Hi,
 
Recently i was in a situation where i need to take the dump of particular records in mysql. We are pretty much familiar with the normal mysqldump statement given below :

mysqldump --user=usr --password=pwd database_name table_name > dumpFile.sql;

The above statement will provide us the dump of the respective database and table into the dumpFile.sql. But if you open the dump file you will see something like this :
 

    //dumpFile.sql
DROP TABLE IF EXISTS `table_name`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `table_name` (
  ...
) ENGINE=InnoDB AUTO_INCREMENT=204182 DEFAULT CHARSET=latin1;

--
-- Dumping data for table `table_name`

LOCK TABLES `table_name` WRITE;
/*!40000 ALTER TABLE `table_name` DISABLE KEYS */;
INSERT INTO `table_name` VALUES ('1','data1');
/*!40000 ALTER TABLE `table_name` ENABLE KEYS */;
UNLOCK TABLES;

Because of the above dump, it will always first drop and recreate the table before sourcing your dump into the database.
So our first task to remove the creation and drop of table from the mysqldump, because when we take dump of particular records, What we want is just inserting the records in database without create and drop of tables and no loss of existing data.
 

    //Creating Dump without create and drop of table :
mysqldump --user=usr --password=pwd --no-create-info database_name table_name > dumpFile.sql;

If you compare the first and above statement there is only one change, –no-create-info option :
It don’t write CREATE TABLE statements that re-create each dumped table. So our dumpFile.sql now contains only this :

--
-- Dumping data for table `table_name`

LOCK TABLES `table_name` WRITE;
/*!40000 ALTER TABLE `table_name` DISABLE KEYS */;
INSERT INTO `table_name` VALUES ('1','data1');
/*!40000 ALTER TABLE `table_name` ENABLE KEYS */;
UNLOCK TABLES;

 

    //Creating Dump of particular records using conditional clause and without create and drop of table :
mysqldump --user=usr --password=pwd --no-create-info database_name table_name --where="id in (1,2,3)"> dumpFile.sql;

In above statement with –no-create-info, we removed the create-drop of table and by –where option we can specify our condition on basis of which we want to fetch the particular records. In above statement it will fetch the record in table that has value 1, 2 or 3 in field ‘id’. So our dumpFile.sql now contains only this :

--
-- Dumping data for table `table_name`
--
-- WHERE:  id in (1,2,3)

LOCK TABLES `table_name` WRITE;
/*!40000 ALTER TABLE `table_name` DISABLE KEYS */;
INSERT INTO `table_name` VALUES ('1','data1'),('2','data2'),('3','data3');
/*!40000 ALTER TABLE `table_name` ENABLE KEYS */;
UNLOCK TABLES;

 
I hope it helps. :)
 
Thanks,
Tarun Pareek
tarun@intelligrape.com
http://in.linkedin.com/in/tarunpareek
More Blogs by Me

Posted in Database

Dbconsole in Grails.

Posted by on December 25th, 2011

So, Grails 2.0 was released a few days back and I upgraded my application to it as soon as I came to know of its final release and it rocks.!!

Among the many things which are making a lot of noise on grails 2.0, there seems to be a lack of noise over the GUI Database console which grails has provided. Developers can connect to the database from a GUI right inside there application, see the data in tables and execute simple queries. Probably its not polished yet so that could be a reason why there is not much noise over it but it still is a usable feature.

All you need to do is, run your grails 2.0 app navigate to http://localhost:8080/app-name/dbconsole/ and configure your database connection. Just select what database you are using, Select the driver, a couple of entries for username and password and you are good to go.

The query builder isn’t too great, but that is no reason to miss this useful tool. No need to move to any other application or console to see whats in the database. The database is available right in your application, and its available only in development mode, so no worries of this slipping into production. Hope this saves some time for all of us.

Thanks.
Sachin Anand
mail : sachin[at]intelligrape[dot]com
twitter : @sachin__anand

Posted in Database, Grails

Truncate Table ? executeUpdate OR createSQLQuery

Posted by on April 14th, 2011

In one of the modules that I was working on , I needed to delete the previous data from a table then load up the data into those tables again and then manipulate code through the ‘id’s’ on those tables.
So this is what I was doing …

Event.executeUpdate('delete from Event')
EventInstance.executeUpdate('delete from EventInstance')



But the problem was that when I deleted the events, the ‘id’s’ on those tables would remain intact and the new Events that would load up would continue upon the earlier present ‘ id’s ‘ .i.e if the last present id was 24 before deleting , the new load-ups would continue from 25 onwards..

This made realise that truncate was a better option … But unfortunately the GORM layer does not support the ‘ truncate ‘ queries that I was looking for.
So in effect this query would give an error

Event.executeUpdate('truncate table Event')
EventInstance.executeUpdate('truncate table EventInstance')

So with some help from ‘ Mr. Google ‘ and my fellow colleagues , i realised that there was another way to do this ..
Session Factory …
Session Factory allowed me to truncate those tables through  createSQLQuery() method
So my new approach was

   def sessionFactory

   def session = sessionFactory.getCurrentSession()
   Query query = session.createSQLQuery('truncate table event')

The getCurrentSession() method looks at the current context to see if a session is stored in there. If there is one, it uses it, and if there isn’t a session in the current context, it creates a new one and saves it in there.
Then all I did was use the query to truncate the tables… and presto !! .. my work was done ..

Hope this helps .. :D

Cheers

Manoj Mohan
Manoj (at) Intelligrape (dot) com

Posted in Grails

Database Backup Script for windows

Posted by on December 14th, 2010

In one of the project I am working on, the application needs to be deployed on windows server. To take the database backup, I wrote the script which does the following :

  1. Takes database dump and copy it to the backup folder.
  2. Zip that backup file and rename it to the format “YYYYMMDD_HHMMSS”
  3. Removes  all the backups older than 30 days.

I used mysqldump command to take database backup. To zip the backup file using the command line, I found 7-zip (download) which is freely available.  We need to have forfiles.exe (download) which enables us to remove old backup files (say older than 30 days).

@echo off
CLS
set hour=%time:~0,2%
if "%hour:~0,1%" == " " set hour=0%hour:~1,1%
set min=%time:~3,2%
if "%min:~0,1%" == " " set min=0%min:~1,1%
set secs=%time:~6,2%
if "%secs:~0,1%" == " " set secs=0%secs:~1,1%
set year=%date:~-4%
set month=%date:~3,2%
if "%month:~0,1%" == " " set month=0%month:~1,1%
set day=%date:~0,2%
if "%day:~0,1%" == " " set day=0%day:~1,1%
set datetimef=%year%%month%%day%_%hour%%min%%secs%
"MYSQL_BIN_PATH\mysqldump" --user=root --password=MY_PASSWORD MY_DATABASE_NAME > "BACKUP_FOLDER_PATH\backup.sql"
"7ZIP_EXE_PATH\7z" a -tzip "BACKUP_FOLDER_PATH\"%datetimef%".zip" "BACKUP_FOLDER_PATH\backup.sql"
del "BACKUP_FOLDER_PATH\backup.sql"
"FORFILES_EXE_PATH\forfiles.exe" /p BACKUP_FOLDER_PATH /s /m *.* /d -30 /c "cmd /c del /q @path

Prerequisites for the above script to work would be :-

  1. Mysql and 7zip installed.
  2. Forfiles.exe is downloaded
  3. All the Path’s in script are replaced correctly.

I also used a background job to run this script automatically every hour on the working days. Though same can be achieved with windows scheduler too.

Hope this helped.

Cheers!
~~Amit Jain~~
amit@intelligrape.com

http://intelligrape.com/

Posted in Database

Grails Liquibase plugin: dbDiff tool workaround

Posted by on October 12th, 2010

The current project I am working on, is going through QA. At the same time development of the new features and bug fixing is on and we couldn’t afford to loose the test data. So synchronizing the state of the QA database with the development was becoming a pain. So we decided to use grails liquibase plugin.

I found one great article by Jackob Kulzer. As explained by Jackob Kulzer, dbDiff tool in the plugin is hard coded to compare development and test environment databases. The other option available was to write every changeset manually. But I was afraid, what if I forget to update even a single changeset.

I decided to write the script, which would overwrite my test database with the schema of the QA database. QA database was on another server. So I had to write multiple scripts to make it work. But once done it made my job really easy.

Following are the scripts to be created:

1. qaSchemaDump.sh (To be created on the development machine):


#SSH to the server and execute generateSchema shell script.
#Note: I had created password less login to the server.
ssh amit@qa.myserver.net /home/amit/Scripts/generateSchema.sh

#Copy QA Database schema to the development machine
scp apg@qa.myserver.net:/home/amit/Scripts/qaDBSchema.sql /home/amit/Scripts/liquibase/qaDBSchema.sql

#Import QA Database schema to test database using createTestDBFromQASchema.sql
mysql -u root --password=myPassword < /home/amit/Scripts/liquibase/createTestDBFromQASchema.sql
exit

2. generateSchema.sh (To be created on the Server):


/* Creates QA database schema */
mysqldump --no-data --tables -u root --password=myPassword myQADatabase > /home/amit/Scripts/qaDBSchema.sql

3. createTestDBFromQASchema.sql (To be created on the development machine):

/* drop, create and then use the test database.*/
drop database myTestDB; create database myTestDB;
use myTestDB;

/*Import the QA database schema to the newly created test database*/
source /home/amit/Scripts/liquibase/qaDBSchema.sql
exit

Once all the above mentioned scripts are created, we just need to run ‘qaSchemaDump.sh’ and followed by ‘grails dbDiff’ and the changeset would be available to us on the console. Then copy this changeset, append it to changeLog.xml and commit it to the repository.

With all this done, login to your QA/production server, take database backup and now we can run grails migrate-sql to confirm the sql that would be generated and then finally run ‘grails migrate’ to update QA/Production database.

Thanks to Jackob Kulzer and Nathan Voxland (author of the plugin)

Hope this helped!

~~Amit Jain~~
amit@intelligrape.com

http://www.IntelliGrape.com/

Posted in Database, Grails

Dumping a Mysql Database

Posted by on April 16th, 2009

In this post i will try to explain how to dump a mysql database so that it can be migrated with same state.In my project i face this situation every week and thanks to dumpfiles that they save lot of time and make the process whole lot easier.

What is a dumpfile ?

A dumpfile is a sql script which contains the step by step instruction to generate a database. It contains the steps to create the table structure and also populates the tables with the original values. By steps i mean the plain simple sql queries.

The Mysql command used to  dump a database is below:

gaurav@gaurav:~$ mysqldump -u root -p   &gt;

e.g. Suppose we have a database named james and we want to take it’s dump.

gaurav@gaurav:~$ mysqldump -u root -p james &gt; ~/Desktop/jamesDump.sql
Enter Password: ******

Now we are back to prompt. We can check the dumpfile by going to the location where it was saved. In the next post I will describe how to use a dumpfile to create a new database.

Hope it helps.

Gaurav Chauhan
gaurav@intelligrape.com

http://www.IntelliGrape.com/

Posted in Database