#60 ✓resolved
Jan Berkel

using rest_mixin kills performance

Reported by Jan Berkel | August 1st, 2009 @ 07:16 PM | in 0.3.2


   def initialize(*args)

  # Explicitly index the classname of a node (required for <code>GET /nodes/MyClass</code>
  # Lucene search to work).
  self.class.indexer.on_property_changed(self, 'classname')   # TODO reuse the event_handler instead !
  # This caused the replication_spec.rb to fail
 # Neo4j.event_handler.property_changed(self, 'classname', '', self.class.to_s)

the on_property_changed line above causes all loaded instances to be committed to the index, regardless if they got changed or not. when using a disk-based index this drastically slows down execution.

self.class.index "classname" might work?

Comments and changes to this ticket

  • Andreas Ronge

    Andreas Ronge August 1st, 2009 @ 10:55 PM

    Hmm, I have never understood why it is needed to add an index on the classname when using the RestMixin since there is only one lucene index file per class (and its subclasses).

    I would like to remove this line
    self.class.indexer.on_property_changed(self, 'classname') or get an explanation why this is needed.

    If it is needed then we should only add this index on created nodes. The initialize method is called when a node is being loaded as well as when a new node is being created. To only create an index when a node is created then the method "init_with_node" should be overridden instead of the initialize method.

  • Martin Kleppmann

    Martin Kleppmann August 1st, 2009 @ 11:21 PM

    Apologies for the classname index -- it's a hack, and I didn't realise it had such a bad performance impact. I put it in there because I needed some way of listing all nodes of a particular class name. The get '/nodes/classname' action does this by calling RestMixin::ClassMethods#find with no search parameters:


    A while ago I think there was a NodeMixin::ClassMethods#all method which retrieved all nodes of a class type by linking them to the reference node and using a traverser. That would be a much better way than using the index. Would it be worth reviving the #all method?

  • Andreas Ronge

    Andreas Ronge August 1st, 2009 @ 11:51 PM

    The all method has moved to the reindexer extension (Yes, I know it's not an obvious name).
    Just require that extension and you will have that all method as well as the reindex method (which uses the all method).
    Sounds like a better solution by using the all method instead of the lucene index.

  • Martin Kleppmann

    Martin Kleppmann August 3rd, 2009 @ 06:54 PM

    I'm working on it -- it's a bit tricky but should work. (Andreas, if you want you can add me as a project member so that the ticket can be assigned to me.)

  • Martin Kleppmann

    Martin Kleppmann August 3rd, 2009 @ 07:12 PM

    • Assigned user changed from “Andreas Ronge” to “Martin Kleppmann”
  • Martin Kleppmann

    Martin Kleppmann August 3rd, 2009 @ 08:53 PM

    • Assigned user changed from “Martin Kleppmann” to “Andreas Ronge”

    I've pushed my proposed patch to Github: http://github.com/ept/neo4j/commit/fe137aa843dfa082254dc3503d2280a3...

    Specs pass. Andreas, could you sanity-check it please?

  • Martin Kleppmann

    Martin Kleppmann August 3rd, 2009 @ 11:23 PM

    • State changed from “new” to “resolved”

    (from [fe137aa843dfa082254dc3503d2280a35e3d972c]) RestMixin no longer requires Lucene index on classname [#60 state:resolved] http://github.com/andreasronge/neo4j/commit/fe137aa843dfa082254dc35...

  • Andreas Ronge

    Andreas Ronge August 3rd, 2009 @ 11:32 PM

    Thanks for the fix. Looks good.

    It would be cool if we could move this rest search functionality to the core Neo4j::NodeMixin.
    I think it's possible for neo4j.rb to decide if a lucene query or a neo4j traversal should be done instead
    of checking if the 'search' parameter exists in the request.
    If one instead parse the query and check if there are lucene indexes for the query then a lucene query will be performed
    otherwise it will traverse. Maybe it is a bit too "magical" ...

  • Andreas Ronge

    Andreas Ronge August 3rd, 2009 @ 11:41 PM

    • Milestone set to 0.3.2
  • Martin Kleppmann

    Martin Kleppmann August 3rd, 2009 @ 11:47 PM

    Sure, I'm happy for it to be in the core NodeMixin if you want -- I originally left it in the REST extension so that it would be less intrusive. Up to you to decide what goes into core :)

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

Neo4j.rb is a graph database framework for JRuby.

It provides:

* Mapping of ruby objects to nodes in networks rather than in tables.
* Storage of ruby object to a file system.
* Fast traversal of relationships between nodes in a hugh node space.
* Transaction with rollbacks support.
* Indexing and quering of ruby objects.
* Integration with Rails

Referenced by