Menu
▾
▴
sqlobject-discuss
|
From: Andrew T. <at...@ga...> - 2013-05-22 23:46:48
|
I recently upgraded an application to the newest version of SQLObject (previously it was using a version from early 2012). This app was caching SQLObject objects in Redis to avoid querying the database for commonly used data. But after upgrading, a bug fix released as part of SQLObject 1.2.3 broke the caching after running into unpickling errors. With the new version of SQLObject, it was refusing to unpickle objects that were already in SQLObject's cache. An example of the error message I received is below (which comes from main.py:1692) eg. "Cannot unpickle User row with id=1 - a different instance with the id already exists in the cache" I've already worked around this to keep my caching working but I was wondering what the bug was that was fixed in 1.2.3 that necessitated this change. The site only says "Fixed a bug in cache handling while unpickling." which I found ironic because it broke caching in my app. Was the bug a memory leak due to having multiple instances with the same ID in memory or was there something else? I ask because my app never seemed to experience any adverse side effects from taking advantage of this "bug". Andrew |
|
From: Oleg B. <ph...@ph...> - 2013-05-23 00:05:58
|
On Wed, May 22, 2013 at 05:46:40PM -0600, Andrew Trusty <at...@ga...> wrote: > I recently upgraded an application to the newest version of SQLObject > (previously it was using a version from early 2012). This app was caching > SQLObject objects in Redis to avoid querying the database for commonly used > data. But after upgrading, a bug fix released as part of SQLObject 1.2.3 > broke the caching after running into unpickling errors. With the new > version of SQLObject, it was refusing to unpickle objects that were already > in SQLObject's cache. An example of the error message I received is below > (which comes from main.py:1692) > > eg. "Cannot unpickle User row with id=1 - a different instance with the id > already exists in the cache" > > > I've already worked around this to keep my caching working but I was > wondering what the bug was that was fixed in 1.2.3 that necessitated this > change. The site only says "Fixed a bug in cache handling while > unpickling." which I found ironic because it broke caching in my app. Was > the bug a memory leak due to having multiple instances with the same ID in > memory or was there something else? I ask because my app never seemed to > experience any adverse side effects from taking advantage of this "bug". This patch, I think: http://sourceforge.net/mailarchive/forum.php?thread_name=E1SXzWF-0005Cg-9K%40webwareforpython.org&forum_name=sqlobject-cvs The last line was not changed but id was removed from the code, that how the last line became broken. Oleg. -- Oleg Broytman http://phdru.name/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
|
From: Andrew T. <at...@ga...> - 2013-05-23 00:16:28
|
I think you just answered how the bug I reported off-list occurred but I was asking about the patch from revision 4530 to 4531 (when the "Cannot unpickle" ValueError was first introduced) that fixed a bug relating to unpickling. On Wed, May 22, 2013 at 6:05 PM, Oleg Broytman <ph...@ph...> wrote: > On Wed, May 22, 2013 at 05:46:40PM -0600, Andrew Trusty < > at...@ga...> wrote: > > I recently upgraded an application to the newest version of SQLObject > > (previously it was using a version from early 2012). This app was > caching > > SQLObject objects in Redis to avoid querying the database for commonly > used > > data. But after upgrading, a bug fix released as part of SQLObject 1.2.3 > > broke the caching after running into unpickling errors. With the new > > version of SQLObject, it was refusing to unpickle objects that were > already > > in SQLObject's cache. An example of the error message I received is > below > > (which comes from main.py:1692) > > > > eg. "Cannot unpickle User row with id=1 - a different instance with the > id > > already exists in the cache" > > > > > > I've already worked around this to keep my caching working but I was > > wondering what the bug was that was fixed in 1.2.3 that necessitated this > > change. The site only says "Fixed a bug in cache handling while > > unpickling." which I found ironic because it broke caching in my app. Was > > the bug a memory leak due to having multiple instances with the same ID > in > > memory or was there something else? I ask because my app never seemed to > > experience any adverse side effects from taking advantage of this "bug". > > This patch, I think: > > http://sourceforge.net/mailarchive/forum.php?thread_name=E1SXzWF-0005Cg-9K%40webwareforpython.org&forum_name=sqlobject-cvs > The last line was not changed but id was removed from the code, that > how the last line became broken. > > Oleg. > -- > Oleg Broytman http://phdru.name/ ph...@ph... > Programmers don't die, they just GOSUB without RETURN. > > > ------------------------------------------------------------------------------ > Try New Relic Now & We'll Send You this Cool Shirt > New Relic is the only SaaS-based application performance monitoring service > that delivers powerful full stack analytics. Optimize and monitor your > browser, app, & servers with just a few lines of code. Try New Relic > and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may > _______________________________________________ > sqlobject-discuss mailing list > sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss > |
|
From: Oleg B. <ph...@ph...> - 2013-05-23 09:48:57
|
Please avoid top-posting. You know the problem: A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? My answer below. On Wed, May 22, 2013 at 06:16:20PM -0600, Andrew Trusty <at...@ga...> wrote: > I think you just answered how the bug I reported off-list occurred but I > was asking about the patch from revision 4530 to 4531 (when the "Cannot > unpickle" ValueError was first introduced) that fixed a bug relating to > unpickling. > > On Wed, May 22, 2013 at 6:05 PM, Oleg Broytman <ph...@ph...> wrote: > > On Wed, May 22, 2013 at 05:46:40PM -0600, Andrew Trusty < > > at...@ga...> wrote: > > > I was > > > wondering what the bug was that was fixed in 1.2.3 that necessitated this > > > change. The site only says "Fixed a bug in cache handling while > > > unpickling." which I found ironic because it broke caching in my app. Was > > > the bug a memory leak due to having multiple instances with the same ID > > in > > > memory or was there something else? I ask because my app never seemed to > > > experience any adverse side effects from taking advantage of this "bug". > > > > This patch, I think: > > > > http://sourceforge.net/mailarchive/forum.php?thread_name=E1SXzWF-0005Cg-9K%40webwareforpython.org&forum_name=sqlobject-cvs > > The last line was not changed but id was removed from the code, that > > how the last line became broken. I think the idea is to prevent getting two rows with the same id via SELECT and unpickling. Oleg. -- Oleg Broytman http://phdru.name/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
|
From: Oleg B. <ph...@ph...> - 2013-05-26 16:12:26
|
Hello! On Thu, May 23, 2013 at 01:48:42PM +0400, Oleg Broytman <ph...@ph...> wrote: > On Wed, May 22, 2013 at 06:16:20PM -0600, Andrew Trusty <at...@ga...> wrote: > > I think you just answered how the bug I reported off-list occurred but I > > was asking about the patch from revision 4530 to 4531 (when the "Cannot > > unpickle" ValueError was first introduced) that fixed a bug relating to > > unpickling. > > > > > http://sourceforge.net/mailarchive/forum.php?thread_name=E1SXzWF-0005Cg-9K%40webwareforpython.org&forum_name=sqlobject-cvs > > > The last line was not changed but id was removed from the code, that > > > how the last line became broken. > > I think the idea is to prevent getting two rows with the same id via > SELECT and unpickling. I fixed the problem but it required much more than just fixing an issue with self.id. The worse problem was that every SQLObject instance has a weak proxy to itself and pickling does strange things -- it calls __getstate__ twice on pickling and calls __setstate__ twice on unpickling. The second call to __setstate__ is certainly a source of problems with cache. Perhaps it's a bug in Python, I'm not sure. I fixed the problem by avoiding pickling the weak proxy and restoring the proxy on unpickling. Committed in the revisions 4601, 4602 (branch 1.3), 4603 (1.4), 4604 (the trunk). I will do releases in a few hours. Thank you very much for helping to find and fix he problems! Oleg. -- Oleg Broytman http://phdru.name/ ph...@ph... Programmers don't die, they just GOSUB without RETURN. |
