Roundup Tracker - Issues

Issue 2551256

classification
consider where __slots__ may be useful
Type: resource usage Severity: normal
Components: Versions:
process
Status: new
:
: : rouilj
Priority: low : Effort-Medium

Created on 2023-01-22 20:51 by rouilj, last changed 2024-06-27 15:46 by rouilj.

Messages
msg7709 Author: [hidden] (rouilj) Date: 2023-01-22 20:51
It looks like using __slots__ can speed up creation and reduce memory use when instantiating 
multiple objects. I wonder if instantiation of hyperdb objects (_Type, Integer, Number, 
String, _Pointer, Password....) could benefit from this.

They seem to meet the criterial: lots of instantiation, limited/no need for dynamic 
properties, and few properties (< 15) to distribute the overhead cost of a dict over.
Also they all inherit from object (so should work for python2) and do not have multiple
inheritance, which should make reasoning about it easier.

Other possibilities are Proptree (seems to be a hot spot in index retrieval esp. for
anydbm) and Class.

Database IIRC can have dynamic properties added to it (e.g. to allow auditors and reactors to 
communicate) so it is not a candidate for __slot__ing.


Issues:

  how to determine how much memory/time is spent instantiating hyperdb objects currently
    can current regression tests be used to determine this, or are they too small
        in size to be useful
  verify that no dynamic properties are added for these classes
  can interface.py be used to add dynamic properties to the classes? If so how to detect
     at compile time or how to tell admins how to detect/fix.

  compare __dict__ definition to __slot__ implementation to see if there is a
    performance and memory use improvement

  how much improvement is needed to make this worth doing for all classes
      maybe to start target 30% reduced memory and/or 20% performance improvement
      (which depends on how often the attributes are accessed).

  this could also cut down on issues if we have typo's in properties, but I consider
      that to be a beneficial side effect and not a goal.
 
references:


  https://betterprogramming.pub/optimize-your-python-programs-for-free-with-slots-4ff4e1611d9d
  https://stackoverflow.com/questions/472000/usage-of-slots
msg8095 Author: [hidden] (rouilj) Date: 2024-06-27 15:46
One more reference:

 https://pythonspeed.com/articles/python-object-memory/
History
Date User Action Args
2024-06-27 15:46:26rouiljsetmessages: + msg8095
2023-01-22 20:51:53rouiljcreate