Spring-ROO and Spring Security, Tutorial 0

Update 2010-08-08 The source code to this series can be found here

I have been struggling with spring security for some time now. It seems that all project need a complete security solution of their own and most of them assume that its some thing that can be solved in rather short time. Generally I would agree. Security is some thing which done right is solvable in very short time with tried and tested libraries. But if the library is a memoth of configuration then you will start running into dead ends. I have been moaning about lack of Spring security features in Spring-ROO, but that is some thing which is very easily solvable as an addon and with 1.1.M3 this should be doable easily with OSGi bundles, but some one still has to do it. I will give it a shot but I cannot promise any thing with my GSoC and other activities.

So instead I will post a series of Tutorials geared towards beginning ROO users to incorporate security into their application.

The first thing I want to say before I start going into any code is to clarify to the new user that ROO is not a framework, it wont do every thing for you, you will have to learn the underlying components of Spring, Spring MVC and other libraries (like Spring Security, ActiveMQ) and tools (like maven). Ofcourse you need to be fairly comfortable with Java as a language. I will not recommend ROO (or for that matter Spring) to any one who is not OK with Java code in general. Second is that an observation, Spring team is really great with documentation, but the more documentation there is, the better chance people have of making their code glue together. In spring security space I highly recommend Spring Security 3 book.

Now lets get started.

So first thing you probably want to do is to create a ROO project, create a POJO called SystemUser (not User because its a reserved word in SQL tables, you can always change the text bundle of your entity so it says User, even if it is called SystemUser). For that you need to create a folder (call it tutorial) and fire up your ROO shell (I prefer the git build :)

project --topLevelPackage com.hatimonline.roosec --projectName tutorial
persistence setup --provider HIBERNATE --database H2_IN_MEMORY
entity --class com.hatimonline.roosec.domain.SystemUser  --testAutomatically
field string --fieldName username --notNull
field string --fieldName password --notNull
logging setup --level DEBUG
controller all --package ~.web
perform tests  (for some reaons this is failing on my box right now)
perform eclipse

So if all goes well, in the above roo script we just created an entity, system user entity which has a username and password. Both are strings, nothing out of the ordinary. You can test the application by doing a

mvn tomcat:run from bash shell.

The first thing you will want is to make username unique, this is simple as putting one annotation in the SystemUser.java file. I recomment that you import your project in Eclipse (or even better STS). I usually prefer to disable the ROO support in STS since I use git based roo-dev. If you have STS then after importing your project, you will bed asked if AspectJ support should be enabled (and would require a restart). Just wait for the application to import every thing (see bottom right hand corner) and then click Yes. STS will restart. From there on hist Ctrl + Shift + R and type partially SystemUser.java (you should be able to select it and start editing it). You can let you ROO shell run outside of eclipse meanwhile.

Add the following on top of usename field.

@Column(unique=true)
@Size(min = 5, max=30)

Notice that the Size annotation could also have been added via ROO. At this point you should be missing some import files, do a Ctrl + Shift + O to organize imports.

Start your tomcat server again. Try to save a username appadmin into the application, it should work the first time. If you try to do it again it would now fail with a  ConstraintViolationException. This is expected because we tried to add to a unique  column a non-unique value. So how can you avoid this. One way is to handle this in the Controller.

Add the following in your SystemUserController.java file (remember the shortcut Ctrl + Shift + R)

@Autowired
private Validator systemUserValidator;

@InitBinder

protected void initBinder(WebDataBinder binder) {

binder.setValidator(systemUserValidator);

}

So what this snippet is doing. It’s autowiring a validator called SystemUserValidator and settig it in you initBinder (I would encourage you to see details in Spring-MVC section of Spring documentation)

At this point you need to create the validator which is autowired. I have skipped the import and package statement for brevity. You may like to put this in a validator subpackage of you source.

@Component

public class SystemUserValidator extends LocalValidatorFactoryBean implements Validator {

private static final Log logger = LogFactory.getLog(SystemUserValidator.class);

@Override
 public boolean supports(Class<?> clazz) {
  return SystemUser.class.isAssignableFrom(clazz);
}

@Override
public void validate(Object target, Errors errors) {
 super.validate(target, errors);
 SystemUser user = (SystemUser) target;

  if (user != null) {
   rrors.rejectValue("username", "The username '"+user.getUsername()+"' is already in use", "The username '"+user.getUsername()+"' is already in use");
   }
  else {
   errors.reject("SystemUser object not available");
  }
 }
}

If you follow the code you will see that this will always result in validation failing. This is not some thing you want to happen. So let’s fix that. One solution that I came up with was to put in a ROO based dynamic finder for SystemUser

finder add --finderName findSystemUsersByUsername --class com.tinyisv.tdca.domain.SystemUser

This will add the capability to search for a SystemUser entity from your database. Once done that you can now add an if condition before you return the error in the validator. This is rather ugly, please help me get a better if statement

	if( SystemUser.findSystemUsersByUsername(user.getUsername()).getResultList().size() > 0 )

If you had roo shell running, you will already get a controller placed for the finder we generated above. The validation should be working now (but as we will see that we need to refine it further if we want to be able to edit our username for a persisted user, as the current condition wont let us persist the same username again)

You will also notice that if you have JSR303 annotations on your entity, their validation sould also work, thanks to @Valid.

This is all for the current tutorial. I am planning to extend the tutorial in the following direction.

  • Actually add spring security, boot strap an admin user and put in UserDetails Service
  • Add more fields and their validation
  • Add support for basic Auditing using AOP (who created, and who last modified SystemUser or any entity)
  • Add ACL schema so as to create a hieracrhy of users who can have varied permissions

My goal was to learn spring security on top of ROO and share it with others. I hope that some one much better than me is already working on a spring-roo security addon. If not then I might just give it a shot in a coming few weeks.

I will also try to share the project code on github, right now it is straight forward enough. Please comment and let me know if you have any improvements. I am still just a student :)

Leave the first comment

Upcoming Java/Spring Conferences and Books

I thought I share some of the conferences and books I am looking forward to in the coming few months

Conferences
This will be the first time I would be heading to developer conferences within EU. This year I had plans to attend FOSDEM, but they did not materilize, there is always next year.

Devoxx: This conference is an Antwerp, a city which I had the pleasure of staying in summer of 2009. This conference is definitely on the top of my list.
JAOO: is a Danish Java conference in Arahus (roughly 4 hours away from where I live by Ferry and Train). I have already put my name on the Volunteers list for this conference
JavaZone: A java conference held in Oslo. Interestingly I found about this by watching a funny movie trailer for this conference just this weekend. I think the very first time I watched a  video of a conference back in 2007 was when i was reseaching framework selection and watched  Matt Raible talking about frameworks in this conference in 2007. Oslo is also 4 hours away from where I live.

Unfortunately my budget wont let me permit to go to Oredev, but I think I have my schedule already full :)

Books
Although I am a strong proponent of learning by practice and experience, one cannot ignore that learning technology from books released in a timely manner is probably the best resource for developers.
I am very interested in the following books. You must be able to tell that this is very Spring centric list :)

Spring in a Nutshell
Enterprise OSGi in Action
OSGi in Action
Spring in Action, Third Edition
Spring in Practice 3rd Edition
Spring Integration in Action
Spring Recipes: A Problem-Solution Approach, Second Edition  (already have the beta)
Spring Security 3 (already released, on my reading list)

I think I would be getting access to most of these books online soon, if only I didn’t have to to spend my savings on going to conferences :P . I will try to update this page with more things I come acoress. Let me know if you know of a good conference within EU and upcoming books related to JavaEE.

Leave the first comment

Mifos GSoC : Day 1

I have been too busy to post that I was selected for GSoC 2010 for Mifos project (my initial prposal can be found here). As I had stated in my proposal, It was not possible for me to start my GSoC official coding portion prior to May 31 2010 (due to my thesis/exam). Since now it out of the way i can now start working.

The first thing that I have done so far is to build MIFOS from git (I built it from SVN last time) I was hoping to see some better results as far as time was concerned but on the same setup this has not improved (in fact it’s gone worse)

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] Mifos - Parent ........................................ SUCCESS [3.181s]
[INFO] Mifos - Common ........................................ SUCCESS [29.953s]
[INFO] Mifos - Test Framework ................................ SUCCESS [15.138s]
[INFO] Mifos - Service Interfaces ............................ SUCCESS [6.605s]
[INFO] Mifos - User Interface ................................ SUCCESS [13.709s]
[INFO] Mifos - Application Programming Interface ............. SUCCESS [4.082s]
[INFO] Mifos - Service Provider Interface .................... SUCCESS [1.442s]
[INFO] Mifos - Application ................................... SUCCESS [22:56.547s]
[INFO] Mifos - Acceptance Tests .............................. SUCCESS [10:38.196s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 34 minutes 49 seconds
[INFO] Finished at: Tue Jun 01 05:54:42 CEST 2010
[INFO] Final Memory: 110M/434M
[INFO] ------------------------------------------------------------------------

But at this moment I am not concentrate about the whole build as I need to focus on the task at hand. So instead of a greenfield project I have to work on making current MIFOS codebase cleaner/saner as far as transactions are concerned. The project and it’s plan as chalked by my mentor were discussed on the mailing-list yesterday (I plan on reviewing this document a bit more carefully with code on the side)
One thing which is different from the previous GSoC project is that I am not doing this alone (another student is working on the same project).
I plan on putting in every day so as to get in the habit of completely open development. For rest of today I will probably be scratching my heaad over the HibernateUtils code :P

3 comments so far, add yours

TrackAndRoute : A waste collection management system (thesis)

I have been waiting for this day since January 2010; today was one of the milestones of my thesis work which I started in January 2010 (but I have been working on this project since late December). The idea is simple, it’s a software which uses information collected by various sensors communicating to the system via GPRS and to make use of that information which is vital for business use. This is applied to the demand driven waste collection industry where collecting fill levels from trashcans can save the end users time and money as well as give them the chance to raise the quality of the system. This requirement was floated by Chalmers school of enterprenuership back in Nov 2009, I applied to this project and got accepted for the role in December 2009. The name of the application was coined by me to emphasise the two aspects of it, tracking and routing based on that tracking.

I cannot discuss the idea or the solution in more details due to NDA.  I hope that later I can publish more details in public since I need to write the whole thing in the form of a thesis any way. But I would like to mention is the choice of technology for this project. This project could have been done in almost any platform as the client never had any specific requirement. I proposed JavaEE (Spring/Hibernate) for the following reasons

  • Familiarity (I have been working on Spring/Hibernate for some good time now and I am familiar with how things work)
  • Portability (the solution produced by this stack is highly portable, can be run in any environment based on tweaks, starting from Google App Engine to good old Unix server)
  • Reliability (the systems produced by this stack are under use all over the world in like of Financial, Datawarehousing, corporate IT industries)
  • Tooling (with advent of spring ROO and STS getting free we now have very good tooling support to rapidly prototype enterprise applications, I will talk about my experiences with ROO a bit later)

There are so many other good reasons to use Spring/Hibernate stack and those reasons can be found in various white papers and case studies published by spring source. Overall I am quite happy that I had a chance to learn lots of new things. Now the not so fun part of actually writing a thesis has begun which I have to deliver by the end of this academic year (Aug 2010). As of today I have officially wrapped up my work for the software portion of my thesis, lets see how many bugs popup during the next few months.It has been a nice learning experience so far and I hope to be a part of this project in the future when it goes commercial (in real sense).

Leave the first comment

On Apple, Facebook and Microsoft

I have had a mostly hate/hate relationship with Microsoft. I know that there are wonderful people who work for them but as an organization they are sometimes setting the bar too low for the technology the deliver which in my view is probably hindering humanity as a whole. I mean rather than doing things like you would do on a unix based platform you have to worry about so many other stupid things, I can argue that if all of the planet was standardized on some unix based system, we would probably have more robust systems. But still they do deserve the credit of being the market leader and innovator in lots of areas of technology. I like Windows 7, have heard good things about the new Visual Studio 2010 and am overall impressed with BingMaps and it’s integration of SeaDragon. Microsoft may be just a huge company looking after it’s interests as they just need to defend their title as the “World Leader”. But all of this is not enough for me to commit my self to Microsoft and explore a career using their stuff, why, oh because I am lazy and don’t want to put double the effort on things (usually in my experience, working with any Microsoft solution costs twice or more in terms of time after you get over the superficial initial productivity gains, but I am opinionated and a n00b with M$ stuff). Also I think opensource is the future for humanity, earning huge loads of money is a good idea, but evolving technology in an open way is much more important. Open standards and open technologies are some things which form the very backbone of the internet as we know today.

Being open is good, and so is using open source technologies. But I think the most important thing is to be “Not Evil”. I know I just stole Google’s motto and to some extent I believe that even they are not truly angelic. But these days Evil is not M$, it’s Apple and Facebook in my opinion.

Apple has been involved in lots of questionable activities in the past few weeks alone. They think that they have made it and users will follow them blindly what ever they do. They make good products for a high premium price and don’t necessarily care about the end consumers (like telling people when new iphone will be released so as to milk off from the old versions, controlling all the channels of their distribution to make sure that who ever they like gets a better treatment and who ever they deem un-likeable, regardless of their stature gets kicked out without any explanation). Apple is plain arrogant, they think they know better, they think they can continue mesmerizing the public with their flashy products. There are few gimmicks that could help them in short run, but they need to focus on being less of a control freak.

Facebook is probably even worse than Apple. I think Steve Jobs just thinks that he knows better what users want, but Mark Zukkaburg has no regard for his end users. He thinks he owns them some how. The privacy policy changes of Facebook reminds me of the Animal Farm, it keeps on getting more and more shady and towards the dark side of the force. Then there are these absolute waste of time games which make billions of dollars go to utter waste. What good is a virtual fence for, some uber looser’s way of escaping the real world? I think Farmeville should be banned and no company should be allowed to waste young peoples time and money like this. Leo Laporte deleted his facebook account which has got me thinking I might be doing that too.

So conclusion: for the forseeable future I would not develop for technologies like Apple and Facebook, not because they are not financially attractive, but because they exploit/control people, which is one thing we have learned is not a good idea.

Leave the first comment

Fetching Typo comments

I have moved my blog to my new VPS but I still have to move all the comments which I had on it. I looked around and could not find any plugin/script for doing so I wrote a quick and dirty mysql query to get every thing in a text file (I only have 24 or so comments from my old blog as I got more comments on my blog entries on my facebook page instead. I already sync them manually with the help of this awesome plugin)

So the query is

echo "select contents.title,feedback.email,feedback.author,
feedback.created_at,feedback.body from feedback JOIN contents
ON feedback.article_id=contents.id;"
| mysql -utheuser -pthepassword typo > /tmp/thetextfile.txt;

Mind the username, password and the text file. If you know a better way to import the comments please let me know. I will just copy/paste them one by one from the text file :)

Leave the first comment

MIFOS integration tests hell

Over the last few days I have been going through java build hell. The project in question is a neat micro finance solution called MIFOS. It’s current build is a mix of JSP,SQL Hibernate, Struts glued together with maven. Every thing checks out and is typical. But what is not typical is the testing framework. I think the intent behind extensive tests was well, but the outcome for the devs at least now is a horror story. Every time you do a MIFOS build it goes through a series of extensive integration and unit tests. And I don’t know any apparent way to avoid those tests and it takes awful lot of time (so far without any success on average of 90 mins, I just had a successful build last night and it took whooping 243 minutes, 10 times more than it should in normal cases)

The thing which I was doing wrong from the beginning was to use 64 bit JDK as I had not read the later sections about problems with 64 bit in the install guide. After few failed attempts and messing around with the GC Overhead workaround as well as adding more RAM to MAVEN_OPTS I got to 32 bit JDK and at least I was able to make it till the next step (ie selenium and acceptance tests). selenium

As of now the build gets stuck here and I am not able to make a significant progress.I can finally get a successful build, but its too slow to do any thing. I will be looking into speeding it up.
complete build

I think the whole approach to unit testing needs to be revised in MIFOS. There are probably too many tests in the entire system and they could be modularized. It wont be easy just separating out business logic in a production system, but clear steps need to be taken so as to reduce the build time for Mifos. This is probably the biggest hurdle for MIFOS community, to attract good developers you need a workable codebase and 10+ mins of buildtime is not a workable codebase in my opnion.

I will be looking into making the build run faster on my own system first and then perhaps see what I can do to make the whole testing thing better. Here is the build summary of my latest build

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] Mifos - Parent ........................................ SUCCESS [2.626s]
[INFO] Mifos - Common ........................................ SUCCESS [27.094s]
[INFO] Mifos - Test Framework ................................ SUCCESS [14.122s]
[INFO] Mifos - Service Interfaces ............................ SUCCESS [7.460s]
[INFO] Mifos - User Interface ................................ SUCCESS [13.232s]
[INFO] Mifos - Application Programming Interface ............. SUCCESS [4.165s]
[INFO] Mifos - Service Provider Interface .................... SUCCESS [1.453s]
[INFO] Mifos - Application ................................... SUCCESS [1:36:39.031s]
[INFO] Mifos - Acceptance Tests .............................. SUCCESS [2:25:11.122s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 243 minutes 1 second
[INFO] Finished at: Tue Apr 20 07:36:19 CEST 2010
[INFO] Final Memory: 87M/380M
[INFO] ------------------------------------------------------------------------
8 comments so far, add yours

Ubuntu 10.04 Beta 2 and lengthy install

Thanks to this bug I have not been able to install Windows 7 and any linux in a straight forward way. I decided to do it the hard way instead. I first backed up my Windows 7 installation (oh yes, I purged all my Linux installs a few weeks ago, I had to , don’t ask, but I planned on coming back with Ubuntu 10.04). i used the latest version of trusty CloneZilla for this. After backing up of Windows I installed Ubuntu 10.04 on a logical partition and then later tried to restore the Windows partitions only to find out that I had screwed up the partitions. What i did was that during Linux installation process I created the partitions from scratch (as I could not see any thing, see the bug i mentioned)

The problem might have some thing to do with the fact that when we create partitions in Linux, they dont share boundries, but when Windows makes partitions, they do share some sort of boundaries. I am not entirely sure if it does cause some error, but it surely is not the same. Here is a dump from my fsdisk

 Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      102400    7  HPFS/NTFS
Partition 1 does not end on cylinder boundary.
/dev/sda2              13       21401   171798528    7  HPFS/NTFS
/dev/sda3           21401       38913   140666720+   5  Extended
/dev/sda5           21401       22185     6291456   82  Linux swap / Solaris
/dev/sda6   *       22185       35239   104857600   83  Linux

If you notice, you will see that there are overlaps between Start and End cylinder, this is what Windows does to the drive, This is not the same in Linux.

Any who, after failing to recover windows from the already installed partition, I recovered my Windows installation and was able to get windows working back again (thanks to recovering the complete disk) and then I only recovered partitions for my linux machine. Infact this failed too first time as I didn’t pay attention and let it rewrite the MBR, so third time was the charm with recovering the whole disk of windows (from before i started installing Linux) and recovering the Linux partition from my recent install. Overall it took me a good part of a weekend to get every thing working. Now I just hope that they can resolve this bug soon. I am convinced that it is caused by Windows 7 (I have not had problems like this in a decade since I have used Linux/Windows)

As for Ubuntu 10.04, like usual there are so many new things and it’s gotten slicker. I did have one problem with the system. It seemed to install the Nvidia driver during the install process but could not activate them (even though it said they were active and not in use). A quick look in the forums suggested purging and re-installing the Nvidia drivers. Other than that, I am getting used to the windows controls on the top left instead of the top right, but thats a minor adjustment :)

I hope I wont get the urge to tinker more in the next 6 months till next Ubuntu release.

Leave the first comment

Nginx + PHP-FPM, long recipe for a tuneup

My new blog engine looked and felt all fine except for being an extreme resource hog. I have barely 1 GB of RAM on my virtual server from HostEurope, which is more than enough if you just want to run one under-subscribed site. But I also happen to run Spring’s new TC server for my own projects (one of which I should be unveiling by the end of this month). So I decided to explore on how to make the whole blog lean. The process was not easy. First I am on Ubuntu LTS so only the most stable software is available for tunning. So this means I had to compile lots of things by hand. Second there is no complete instruction set for doing so on Hardy (which will be ousted soon by new LTS version). But some instructions are still worth mentioning. I might be tempted to write my own complete DIY nginx,wordpress on php-fpm entry after the new LTS is released.

I decided to go with Nginx and PHP-FPM solution for running WordPress (and then later I can also put in Passenger for my RoR applications). Nginx is very lean Webserver which is changing the whole landscape of LAMP to LEMP (it’s pronounced as engine-x). I had been using it previously as well but it’s a moving target and has progressed a lot since I last dove into configuration files. There are various resources on the internet addressing this kind of migration but none of them complete. In order to handle redirects as 301 and not as 302 I used this plugin, and this post was extremely helpful in getting the permalinks working properly.

I also got to optimize my MySQL for memory use but I think there is still room for improvement.

Initial tests with a blank TC server and a running blog on stock Apache and Mysql would reat away all the available RAM. Now I have been able to shave off about 100 MB (which is not bad). I will see if running standard Tomcat works as it uses less memory than TCServer (from the looks of it)

Leave the first comment

Moving my olders posts

I just did it, ported my blogger entries to WordPress, all 200+ of them. And it is a mess. I realize that over the years I have learnt how to write better and express my self in a clearer way. This is all thanks to loads of time reading other people’s blogs and some what maturity. But I still have to be much better at writing and communication in general. Importing blogger is built in to WordPress, you just click and authorize. If only it was so easy to port Typosphre blog entries. I copied the RSS feed of my Typo based blog but without any comments. Hopefully I would be able to move them soon as well. It would be nice to get the comments which appear on my Facebook, reappear on word press as well.

I realize that in the past I have probably put out some naive comments (like about how I would never leave Blogger.com or how much I hate a dish called Nargisi Koftas, some thing I won’t mind munching right now or the worst that how I cannot stand those corporate blogs, I really have come a long way) but since it was all already out there, I thought it’s better to organize it all and bring it under one roof.

I have become more aware of what to put on the web and how to put it. It is  fun to see how my personality has changed over the years just by looking at the old blog entries.

Leave the first comment