Thread: [SQLObject] Closing connection

Hello,

I'm looking at improving tests of rebuildd[1].

The test suite create and delete the database[2] for each test in the
setUp()[3] method, but sqlobject don't like it.

The first test is OK but all successives ones fail with "disk I/O error".

I found a thread[4] on the list speaking about the same issue but I do
not manage to fix my tests.

Any hints?

Regards.

Footnotes: 
[1]  http://packages.qa.debian.org/rebuildd

[2]  http://anonscm.debian.org/gitweb/?p=rebuildd/rebuildd.git;a=blob;f=tests/RebuilddTestSetup.py;h=d677d8e481baada9218d9f86687a818c87b28582;hb=HEAD

[3]  http://anonscm.debian.org/gitweb/?p=rebuildd/rebuildd.git;a=blob;f=tests/TestJob.py;h=20e81d6c3de9a50ddb4a95c04593d739f31dbec8;hb=HEAD#l16

[4]  http://thread.gmane.org/gmane.comp.python.sqlobject/5769

-- 
Daniel Dehennin
Récupérer ma clef GPG:
gpg --keyserver pgp.mit.edu --recv-keys 0x6A2540D1

On Wed, Feb 01, 2012 at 02:53:20PM +0100, Daniel Dehennin wrote:
> I found a thread[4] on the list speaking about the same issue
> [4]  http://thread.gmane.org/gmane.comp.python.sqlobject/5769

   The thread has the answer, IMO.

> but I do not manage to fix my tests.

   What's up?

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

Oleg Broytman <ph...@ph...> writes:

> On Wed, Feb 01, 2012 at 02:53:20PM +0100, Daniel Dehennin wrote:
>> I found a thread[4] on the list speaking about the same issue
>> [4]  http://thread.gmane.org/gmane.comp.python.sqlobject/5769
>
>    The thread has the answer, IMO.

So, I should have not understand:

#+begin_src
test_init_job (TestJob.TestJob) ... E: setUp Cannot operate on a closed database.
#+end_src

I move the os.unlink fro the setUp() to tearDown() which call a new function:

#+begin_src python
def rebuildd_global_test_teardown():
        try:
            Rebuildd().sqlconnection.close()
	    sqlobject.sqlhub.processConnection = Rebuildd().sqlconnection
	    sqlobject.dbconnection.TheURIOpener.cachedURIs={}
            os.unlink("/tmp/rebuildd-tests.db")
        except Exception, e:
            print "E: tearDown %s" % e
            pass
#+end_src

Regards.
-- 
Daniel Dehennin
Récupérer ma clef GPG:
gpg --keyserver pgp.mit.edu --recv-keys 0x6A2540D1

On Wed, Feb 01, 2012 at 03:28:30PM +0100, Daniel Dehennin wrote:
>             Rebuildd().sqlconnection.close()

   Aha, got it. You've stumbled upon a subtle bug in SQLiteConnection.
SQLiteConnection uses a different (from its parent class DBAPI)
implementation of .getConnection() and maintain its own pool -
._threadPool along with DBAPI._pool. But it doesn't clear the pool on
.close().

   The quick-and-dirty solution for you is to clear the pool yourself:

            Rebuildd().sqlconnection.close()
            Rebuildd().sqlconnection._threadPool = {}

   I'll add a proper .close() to SQLiteConnection.

> 	    sqlobject.sqlhub.processConnection = Rebuildd().sqlconnection

   You preserve an old SQLiteConnection, so you don't need the
following:

> 	    sqlobject.dbconnection.TheURIOpener.cachedURIs={}

   The workaround is for the case when you want to create a new
SQLiteConnection like this:

            Rebuildd().sqlconnection = connectionForURI(...)

   And, BTW, this

>             os.unlink("/tmp/rebuildd-tests.db")

   is just

            Rebuildd().sqlconnection.dropDatabase()

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            ph...@ph...
           Programmers don't die, they just GOSUB without RETURN.

Oleg Broytman <ph...@ph...> writes:

>    Aha, got it. You've stumbled upon a subtle bug in SQLiteConnection.
> SQLiteConnection uses a different (from its parent class DBAPI)
> implementation of .getConnection() and maintain its own pool -
> ._threadPool along with DBAPI._pool. But it doesn't clear the pool on
> .close().
>
>    The quick-and-dirty solution for you is to clear the pool yourself:
>
>             Rebuildd().sqlconnection.close()
>             Rebuildd().sqlconnection._threadPool = {}

Thanks, this solve my issue.

>    You preserve an old SQLiteConnection, so you don't need the
> following:
>
>> 	    sqlobject.dbconnection.TheURIOpener.cachedURIs={}

Ok, I plan to change rebuildd code and avoid singletons, this will
permit to make each unit test in a separate context (and directory) to
avoid side effects.

At that time, I think I will be freed of all of this.


[...]

>    And, BTW, this
>
>>             os.unlink("/tmp/rebuildd-tests.db")
>
>    is just
>
>             Rebuildd().sqlconnection.dropDatabase()

Thanks a lot.
-- 
Daniel Dehennin
Récupérer ma clef GPG:
gpg --keyserver pgp.mit.edu --recv-keys 0x6A2540D1