DISQUS

DISQUS Hello! /var/log/mind is using DISQUS, a powerful comment system, to manage its comments. Learn more.

Community Page

/var/log/mind

Dhananjay Nene’s free (as in free speech) opinions on all things related to Software Engineering
Jump to original thread »
Author

Turbocharge your string keyed hashmaps

Started by Dhananjay Nene · 10 months ago

This post gives you a small tip which just might make a world of difference to your java hashmap’s performance. This trick has been inspired by the “symbol” construct in Ruby language.

I have often considered using hash maps using Strings as keys as quite exp ... Continue reading »

12 comments

  • How could that be faster when you are just deferring the regular String lookup to the Symbol map and then only slightly optimizing *another* hash lookup. If you make a case to keep a hard reference to the Symbol object outside of the map, then you might as well do that for the String itself and get the same optimization.
  • @Jacob The *another* hash lookup you mention will try to check for the identity and then equality of keys. The class Symbol is constructed such that underlying strings are always identical (even if the symbol is constructed twice using two non-identical but equal strings). This is what makes the big difference.
  • Doesn't String.intern() effectively do what you want?


    Of course, this still seems like a pointless optimization. In my test of doing a million puts/gets, the difference in your two approaches about 100ms, or .104 milliseconds per operation. Not sure that would make a difference in most application contexts....
  • @Dave. Have updated the post with the sources to be able to verify my findings. I missed the Strings.intern() part. I think it will certainly make for a simpler Symbol implementation. However the Symbol implementation will still be required.
  • Looking at your tests, theres a couple issues: 1) you run the 'usual' test first without a JVM warmup-- so that 1% probably shouldn't even be there, I'd even say you'd hit a negative ratio there. 2) You aren't accounting for the Symbol map lookup in your tests and are using a hard reference to the symbols. A real world example of the testSymbol would be to walk through each String key, do a Symbol.get(...) and then do the hashmap lookup.
  • @Jacob. I agree with the non-warming up and the non accounting of symbol map lookup during Symbol construction time. Here's the rationale. Actually I am not so concerned with the timings for the usual (actually I should've called it identical). If a programmer can keep track of ensuring that the strings are identical - either explicitly or by always keeping a cached version of String.intern() result as suggested by Dave then Symbol class isn't really required. The 1% keeps on varying between slightly negative and slightly positive across different runs (basically the additional cost of lookup and the cached value of hashcode are being traded off with each other). Yes the test does not reflect the cost of creation of a symbol. But in most programs, the number of times symbol is created can be made to be much much lower than the number of times it is used for lookups. What you suggest of as a real world example is actually not necessarily a good real world example - since it is quite feasible to ensure that the symbols get created far less often than the getter is called.
  • @Dave. To correct myself, I think if String.intern() is called each time a string key is used, one should get similar results without needing a Symbol class.
  • Well to begin with String being mostly an immutable object computing the hashcode once seems a natural and hashCode does cache the calculated value. The reason why fast code is indeed faster is because the compiler interns String literals http://java.sun.com/j2se/1.4.2/docs/api/java/la...)
    As to the static HashMap in the symbol class it is I am afraid a memory leak being a cache with no eviction strategy and one that can not be flushed, as a bonus getSymbol is not Thread safe! Static is indeed evil http://gbracha.blogspot.com/2008/02/cutting-out...
  • @khalil Addressed your concerns in an updated version.
  • Calling intern can lead to permgen memory exhaustion see this bug issue on Xstream http://jira.codehaus.org/browse/XSTR-395 the fix involves using weak references to get around the cache size issue
  • If performance is the main issue and you are creating a Symbol class anyway, consider putting the associated values directly in Symbol's fields. That will be as fast as it gets.
  • I have many doubts about your optimization:
    1. You should NEVER use new String(String). It forces the JVM to create a new String object.
    2. String literals, such as "mykey", are already automatically interned by Java.
    3. The call to intern is slow, since it performs a lookup in a hashtable containing ALL String literals.
    4. Hashcode is already stored in String, and calculated the first time it is needed (see source code of String.java)

Add New Comment

Comments are closed for this post.
Returning? Login