python class that creates instances with a variable , but with __setattr__ overridden

[Herr.Hoefkens]

this shouls be simple imho but after 5 days of breaking my head against it , i think i need some help.

tbh i dont do OOP, but since python is as OOP as OOP goes under the hood. sometimes you have to...

 

the end goal is to have a metaclass that inherits from type(makes classes)

a baseclass for the metaclass that can serve as a blueprint for spinoffs of that class.

 

every instance that gets created in the end should have this layout:

a regular class instance , with 1 default attribute set, that is an instance attribute, not a class attribute (<= as a class attibute i can get it to work but thats not what i need)

this attribute 'Data' should be a dict

when an instance gets create trough one of the subclasses , they all add one attribute to the instance. . eg if an instance earth is created with the class 'planet' , it calls the constructor in the solarsystem class , that one in the galaxy class eg they would respectively set the attributes planet = earth , solarsystem=ours, galaxy=milkyway on the instance

now  on the instance the following methods should be overridden :

• __getattr__
• __setattr__
• __getitem__
• __setitem__
• __iter__
• __contains__
• __delitem__
• __delattr__
• __hash__
• __len__
• ...

and they should work such that they no longer work on the attributes of the instance but on the one attribute that is the dict=>

foo.something=1

should actually  set foo.Data['something']=1

 

the problem i run into each time is that when i try to set the dict before the instance is created the  dict becomes a class variable, and is shared by all instances , wich should not be the case.

if i try to set it in the __init__ method. the rest of the methods is already overridden , and trying to create instance.Data . actually tries to create instance.Data.Data={} since it calls the setattr methond , and since there is no instance.Data yet it fails... here are some (in the mean time exfiltrated code im trying to get work , if  for starters the get, set of both addr and item work as it should im a happy man. :

 

breakpoint()
class QBase(type):
	def __new__(cls, name, bases, attrs):
		print(f'QBase.__new__ called with {cls}:{name}:{bases}:{attrs}')
		return super().__new__(cls, name, bases, attrs)
	def __init__(s,*a,**k):

		super().__init__(*a,**k)

class QBaseDict(metaclass=QBase):
	def __new__(cls, *args, **kwargs):

		return super().__new__(cls)
	def __init__(s,n,*a,**k):
		print(f'QBaseDict.__init__ called with {s},{n} ')
		s.Data=dict()
		super().__init__(*a,**k)
	def __setattr__(s, k, v):
		super().__setattr__(k,v)


class QDict(QBaseDict):
	def __new__(cls, *args, **kwargs):

		return super().__new__(cls)

	def __init__(s,n,*a,**k):

		super().__init__(n,*a,**k)
	def __setattr__(s, k, v):
		s.Data[k]=v




def test(n):
	t=QDict(n)
	return t

a=test('foo')
b=test('bar')
b.Data['test']='ikkel'
breakpoint()

 

https://replit.com/@hoefkensj/customtypeproblems#main.py

 

[Eigenvektor]

1 hour ago, Herr.Hoefkens said:

the end goal is to have a metaclass that inherits from type(makes classes)

a baseclass for the metaclass that can serve as a blueprint for spinoffs of that class.

This sounds a bit like an XY problem. What are you hoping to achieve with that class? Maybe there's a better way than to redesign the language itself. I'm not a Python developer, but I'm not entirely certain I understand what you're actually trying to achieve from a more general OO point of view.

[Herr.Hoefkens]

i have a solution somewhat :

class QBaseMeta(type):
	def __new__(c, n, b, a):
		c.Type=None
		return super().__new__(c, n, b, a)
	def __init__(s,*a,**k):		super().__init__(*a,**k)

class QBase(metaclass=QBaseMeta):
	def __new__(c,*a,**k):
		Class=super().__new__(c)
		return Class
	def __init__(s,n,*a,**k):
		s.Type=s.__class__.__name__
		s.Data=dict()
		super().__init__(*a,**k)
	def __setattr__(s, k, v):
		if not 'Data' in dir(s):super().__setattr__(k,v)
		else : s.Data[k]=v
	def __getitem__(s,k):		return s.Data[k]
	def __setitem__(s,k,v):		s.Data[k]=v
	def __delitem__(s,k):		del(s.Data[k])
	def __delattr__(s,*a,**k):		del(s.Data[k])
	def __len__(s):	return len(s.Data)


class QDict(QBase):
	def __new__(c,*a,**k):	return super().__new__(c)
	def __init__(s,n,*a,**k):	super().__init__(n,*a,**k)

class QSection(QBase):
	def __new__(c,*a,**k):return super().__new__(c)
	def __init__(s,n,*a,**k):
		s.Section=n
		super().__init__(n,*a,**k)

def fnQDict(n):	return QDict(n)
def fnQsection(n): return QSection(n)

foo=fnQDict('foo')
bar=fnQDict('bar')
bar.Data['Test']='Ikkel'
Qtm=fnQsection('Qtm')
Qtm.Mtd=fnQsection('Mtd')

it works but this cant be the way to do it , this would mean that every time a value is read , two if blocks have to be considered , if blocks who only serve for first run as it were, not to mention the 4 if blocks if a value is set. now think about setting a vallue in a 5 levels deep nested dict, like a ms timer or whatever, now i know you shouldnt use python if speed matters, but i found python always a great language to do GUI and Gui Glue in . but that would become a cpu heavy slow responding gui that way. and the super() was intended to pass trough stuff to built in ctypes instead of  handling it in slow python code

[Herr.Hoefkens]

~~deleted~~

[Herr.Hoefkens]

#!/usr/bin/env python

ncals=0
class QBaseMeta(type):

	def __new__(c, n, b, a):
		c.Type=None
		base = super().__new__(c, n, b, a)
		return base

	def __init__(s,*a,**k):
		super().__init__(*a,**k)

class QBase(metaclass=QBaseMeta):

	def __new__(c,*a,**k):
		Class=super().__new__(c)
		return Class

	def __init__(s,n,*a,**k):
		s.Type=s.__class__.__name__
		s.Data=dict()

		super().__init__(*a,**k)

	def __setattr__(s, key, val):
		global ncals
		ncals+=1
		print(f'\x1b[1Gsetattr IFBLOCK ran {ncals} times!',end='')
		if not 'Data' in dir(s):
			super().__setattr__(key,val)
		else :
			s.Data[key]=val

	def __getitem__(s,key):
		item=s.Data.get(key)
		return item

	def __getattr__(s,key):
		attr=s.Data[key]
		return attr

	def __setitem__(s,key,val):
		s.Data[key]=val

	def __delitem__(s,key):
		del(s.Data[key])

	def __delattr__(s,*a,**k):
		del(s.Data[k])

	def __len__(s):
		lendata=len(s.Data)
		return lendata
	def keys(s):
		print(s.Data.keys())
		return s.Data.keys()
	def values(s):
		return s.Data.values()

class QDict(QBase):
	def __new__(c,*a,**k):
		return super().__new__(c)
	def __init__(s,n,*a,**k):
		super().__init__(n,*a,**k)

class QElement(QDict):
	def __new__(c,*a,**k):
		return super().__new__(c)
	def __init__(s,Name,QType,*a,**k):
		s.QType=QType
		s.QClass='QElement'
		super().__init__(Name,*a,**k)

class QSection(QBase):
	def __new__(c,*a,**k):
		return super().__new__(c)
	def __init__(s,n,*a,**k):
		s.Section=n
		super().__init__(n,*a,**k)

################
## Q&D Tests  ##
QApp=QElement('QApp','QApplication')
QApp.Cfg=QSection('Cfg')
Qtm=QSection('Qtm')
Qtm['Mtd']=QSection('Mtd')

for sect in ['Get','Set','Sig','Wrp']:
	Qtm[sect]=QSection(sect)
	for letter in [chr(x) for x in range(20)]:
		Qtm[sect][letter]=QSection(letter)
QApp.Qtm=Qtm

this kind of works but testing it for this small setup , that ifblock already ran 267 times:

stuff that doesnt work as it should and i seem to go and try figurout trial and error way:
__iter__

__dict__

__contains__ (this one i got i thinnk)

 

seriously , the guy that  ever came up with oop,should be `//=== - - (Oo)<-

C, VHDL, is supposed to be complicated compared to this

made a gui once with pyqt that could explore itself its own structure but i never ran so many times up against an infinitloop as the past 4 days

 

and what i need it for its supposed to hold the sysfs parsed , with structure, the sysfsname, the human understandable name ,but simultaniously the last read of the value, a function that reads the value directly form sysfs, and a function that can write the value, next to it and the path of where the value is, that next to the MSR, and the known bits in the MSR

was doing it with dicts and got allong fine but once your 20 levels  deep in a dict it gets hard to read what path you actually have inside the dict with all the ['name']['othername']...often ending  in ['foo']() wich i miss each time that gets executed  so

 

 

and PS , i know that possibly there is a more convenient way, possibly better way, but im not gonna change my coding style 180degrees , bound to produce bugs and worse .... this might indeed be an Y to my X , but more so a standin Y for my current Y , but also a more permanent solution to use if i can get it to work , that struct will prolly serve as the default datablock(s) for future projects(and possybly even past unfinisched ones some day:)

[NocTheRocc]

To answer your question directly, the issue is that when you overload __setattr__, your attempt to set s.Data will use the overloaded version. To fix this, use the __setattr__ implementation from the parent:

class QDict:
	def __init__(self):
		super().__setattr__("Data", dict())

	def __setattr__(self, k, v):
		self.Data[k]=v

b = QDict()
b.Data['bin']='baz'
b.eggs = 'spam'
print(b.Data) # {'bin': 'baz', 'eggs': 'spam'}

You can then overload other builtin methods to your taste.

 

17 hours ago, Herr.Hoefkens said:

and what i need it for its supposed to hold the sysfs parsed , with structure, the sysfsname, the human understandable name ,but simultaniously the last read of the value, a function that reads the value directly form sysfs, and a function that can write the value, next to it and the path of where the value is, that next to the MSR, and the known bits in the MSR

was doing it with dicts and got allong fine but once your 20 levels  deep in a dict it gets hard to read what path you actually have inside the dict with all the ['name']['othername']...often ending  in ['foo']() wich i miss each time that gets executed  so

If I'm understanding the problem correctly, I do think there's some XY problem going on here. The first thing I'll point out is that wrapping a class around a dictionary is redundant, since objects already behave like dictionaries:

>>> class A:
...     pass
...
>>> a = A()
>>> a.x = 2
>>> a.x
2
>>> setattr(a, "y", 3)
>>> a.y
3
>>> a.__dict__
{'x': 2, 'y': 3}

So a simple way to solve your problem would be to add entry properties to QSection and QElement properties when they're constructed, presumably pulling the entries to populate from the sysfs path that was passed in.

class QSection:
	def __init__(self, path):
		self.path = path
		for entry in path.iterdir():
			if entry.is_dir():
				setattr(self, entry.name, QSection(entry))
			else:
				setattr(self, entry.name, QEntry(entry))

class QEntry:
	def __init__(self, path):
		self.path = path
		# ... populate other entry properties here ...

	def read(self):
		# ...

	def last_value(self):
		# ....

	def write(self, new_value):
		# ....

 

Then you can have something like

root = QSection(Path("/sys/something"))
root.foo.bar.bin.baz.read()

[Herr.Hoefkens]

On 4/16/2023 at 11:45 AM, NocTheRocc said:

To answer your question directly, the issue is that when you overload __setattr__, your attempt to set s.Data will use the overloaded version. To fix this, use the __setattr__ implementation from the parent:

class QDict:
	def __init__(self):
		super().__setattr__("Data", dict())

	def __setattr__(self, k, v):
		self.Data[k]=v

b = QDict()
b.Data['bin']='baz'
b.eggs = 'spam'
print(b.Data) # {'bin': 'baz', 'eggs': 'spam'}

You can then overload other builtin methods to your taste.

 

If I'm understanding the problem correctly, I do think there's some XY problem going on here. The first thing I'll point out is that wrapping a class around a dictionary is redundant, since objects already behave like dictionaries:

>>> class A:
...     pass
...
>>> a = A()
>>> a.x = 2
>>> a.x
2
>>> setattr(a, "y", 3)
>>> a.y
3
>>> a.__dict__
{'x': 2, 'y': 3}

So a simple way to solve your problem would be to add entry properties to QSection and QElement properties when they're constructed, presumably pulling the entries to populate from the sysfs path that was passed in.

class QSection:
	def __init__(self, path):
		self.path = path
		for entry in path.iterdir():
			if entry.is_dir():
				setattr(self, entry.name, QSection(entry))
			else:
				setattr(self, entry.name, QEntry(entry))

class QEntry:
	def __init__(self, path):
		self.path = path
		# ... populate other entry properties here ...

	def read(self):
		# ...

	def last_value(self):
		# ....

	def write(self, new_value):
		# ....

 

Then you can have something like

root = QSection(Path("/sys/something"))
root.foo.bar.bin.baz.read()

yeah your absolutely right about the xy thing i know l, all i need is in fact a dict, and that is how i have done it up until recently. the problem with dicts , and thats not with the format of dicts , because that is good enout , but more with the limits in handling dicts. large portions of my  code started to look like this. ps im a firm believer in the col 80 limit rule , and yes i use tabs with tabwidth 2

Foo['BarBarBar'][foo]|= bar['Foo'][foobar]['rab'][bar](ararg[test]['ikkel'][a].ararg[test]['okkel'][b])
Foo['Bar'][foo]|= test['Foo'][foobar].get('ikkel',default).format(Bar=frm[bar]['ar'],tar=ararg[test]['ikkel'])

and since not all 'keys' are static ones and usually its the second one that is chinging in a for or while loop , followed by, one or two static ones then a changable one again eg

and wanching at my code all i saw was: `[###'']['###'[###]]['###']'[###] ` and it started to become a hazard as it started to happen more and more that i was changing a line, had to quickly look smth up in a different modulel or lower/ higer in the same module and when comming back from it , i edited the wrong line without noticing , and cursing a day later that i cant find it when debugging.

 

so i figured i need a way i can still call the dicts but with less [''] and a way i could filter the dicts out without having to digress into them looking for a key and checking its value for smth... so i was trying to achieve both at once ,  the setup would have allowed me to acess the same dicts as :

 foo.bar[foobar].test.ikkel[variable].fnx(bar.foo.bar, foo.bar.foo)

and i could filter on isinstance(type , dict[key]), or as i origianlly planned to also overwrite the contains method

that wen a type is provided as in

if typea in dict['c']: , that it would not check against the keys but against the values of the keys , wile retaining the other behaviours of to originals (and since a key cant be a class or a type it would not break anything i figured

[Herr.Hoefkens]

hehe , the misery of sysfs is that it is a maze like you have never seen a maze before, the reason im parsing out stuff and reorganizing it in a way that suites the purpose of the app,

i made a dict browser a while back , it does not show the exact structure of the dict but its pretty close, note that the app itself is also made using the same principes and manners of creating dicts ( so its a fully procedural programmed qt gui )

ps thas only the freq and temps of the cpu , nothing more , but its a good small example.

also note the pathbar just below the table , it shows the path in the dict''