Thread: [Sqliteman-devel] Sqliteman future

Hello,

I have been using Sqliteman for quite a while and it is indeed the best
Sqlite manager out there. Lately, I have been thinking of implementing
Sqliteman with an "analytic" part. I have been using Sqlite to store data
for my analysis and I found it really good. My main issue is that I use
python and R coding to make my analysis and I have scripts all around. My
thought is: what if Sqliteman could be implemented with a pluggable analytic
part? So, maybe with the possibility to create ad hoc analysis plug ins with
python.

This is something that could be really useful for me and I would do the
programming myself. However, I only know python programming and have some
basics of qt. I would like to know if someone could be interested in helping
me. I can also do the programming myself, but I need an introduction to
Sqliteman source code and eventually some tips to learn fast C++.

Any ideas?
a.c.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

hi Alessio,

thanks for you interest.

The situation around Sqliteman is not good. There is no progress in it for months. The 1.2.x versions are used still, but the development 1.3.x never touched its testing releases although it contains many improvements.
The main reason is that I don't have free time to work on this software and mainly - I don't need it to my regular work too as there is no sqlite db around me now.

There were some movements how to start its development again but with no (or minor) success.

Maybe you will be the missing startup element? Who knows...

responses to your questions follows:

On Jan 1, 2011 (Saturday), at 5:03 PM, Alessio Civitillo wrote:
> I have been using Sqliteman for quite a while and it is indeed the best Sqlite manager out there. Lately, I have been thinking of implementing Sqliteman with an "analytic" part. I have been using Sqlite to store data for my analysis and I found it really good. My main issue is that I use python and R coding to make my analysis and I have scripts all around. My thought is: what if Sqliteman could be implemented with a pluggable analytic part? So, maybe with the possibility to create ad hoc analysis plug ins with python.

There is no support for 3rd party plugins.
There is no scripting support too.
There is support for sqlite3 extensions in 1.3.x.

There are some solution how to enable scripting in Qt4 app - manual bindings, KROSS (http://techbase.kde.org/Development/Tutorials/Kross/Introduction), and more. But I had never any reason to do it.

As a dummy solution we can add a quick workaround with some user-defined menu entries which can run your scripts with some additional arguments like:
callExternalScript("python foo.py /path/to/databasefile.db providedusername pass...");
and handle its results

> This is something that could be really useful for me and I would do the programming myself. However, I only know python programming and have some basics of qt. I would like to know if someone could be interested in helping me. I can also do the programming myself, but I need an introduction to Sqliteman source code and eventually some tips to learn fast C++. 

Coding in Qt4 is quite easy. It handles most of memory management for you itself like Python does. I can say there is only syntax different between PyQt and pure Qt4 if you will make only simple things like forms, image loadings, or R interpret calling. Please take it as a "motivation and introduction" only ;)

ad Sqliteman code internals: most of the code is (I hope) self explanatory - there are no low-level techniques used. Most code files contain "extended functionality" like various dialogs (new db objects, populator, exports, imports, ...) - it's usually small file with little "blackboxed" functionality.

most of code should be documented, 1.2.x docs are available online:
http://docs.sqliteman.com/devel-doc/annotated.html

main app is located in litemanwindow.* which connects basic GUI together.

Direct data manipulation (without sql) is done in dataviewer.* and its sub-components.

database.* files contains functions to query or manipulate raw SQL

Personally I'm not happy with current architecture of DB transaction handling as it mixes raw SQL access (BEGIN; foo; COMMIT;) with Qt4 QtSql module transactions (in dataviewer). I had plans to write own driver to be used in the app as only db access method one day but there is no motivation for now on my side.
And of course - all DB access is done in main app thread so the app can looks "frozen" in some circumstances.

Anyway, I think it's enough for quickstart. Feel free to ask.

cheers,
petr

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQEcBAEBAgAGBQJNJEwIAAoJEC8yRjM4uE2tNvUH/1fdYSeNP3Sg+Qe5yGlNXzB5
wSYnICpxBT6vCKBsonF1QGPXDFo6z3YWQNCSy23KN+9ODCNDyYnEHBVawysGHpO3
Hw8fRiAP81o7DTiFPdhke3YT6TnE6z4TwNaQjyOM4/Dah3kFzvxgfOpWg72HmDmJ
SHTUgArvZ6BmlYbIkWmvgbMbJNHhKwVSuxRXOxxQT8Tdilih0nCQ9VTt6E5hT0Bc
0pX0mfUWHXPlV3IytpEIfNwSqnqzjULZMXALxpOOWGxm3d0utgWugKpPTeFT6NhO
JNvlClgwOpEtCCpSpJo+nKl2kEqadaNyZK/rWohJajkCoXPY2QrDqwyq70zyy4I=
=pxcD
-----END PGP SIGNATURE-----

I'm among those that committed to its continuity but things held me down and
wasnt able to commit as promised. The project is still in my list of things
to do and I had done quite some bit of redesign and wrote a new UI based on
Qt Creator IDE.

I hope to get some spacing to complete it.


Regards,
Kennedy Kasina

On Wed, Jan 5, 2011 at 1:46 PM, Petr Vanek <pe...@sc...> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> hi Alessio,
>
> thanks for you interest.
>
> The situation around Sqliteman is not good. There is no progress in it for
> months. The 1.2.x versions are used still, but the development 1.3.x never
> touched its testing releases although it contains many improvements.
> The main reason is that I don't have free time to work on this software and
> mainly - I don't need it to my regular work too as there is no sqlite db
> around me now.
>
> There were some movements how to start its development again but with no
> (or minor) success.
>
> Maybe you will be the missing startup element? Who knows...
>
> responses to your questions follows:
>
> On Jan 1, 2011 (Saturday), at 5:03 PM, Alessio Civitillo wrote:
> > I have been using Sqliteman for quite a while and it is indeed the best
> Sqlite manager out there. Lately, I have been thinking of implementing
> Sqliteman with an "analytic" part. I have been using Sqlite to store data
> for my analysis and I found it really good. My main issue is that I use
> python and R coding to make my analysis and I have scripts all around. My
> thought is: what if Sqliteman could be implemented with a pluggable analytic
> part? So, maybe with the possibility to create ad hoc analysis plug ins with
> python.
>
> There is no support for 3rd party plugins.
> There is no scripting support too.
> There is support for sqlite3 extensions in 1.3.x.
>
> There are some solution how to enable scripting in Qt4 app - manual
> bindings, KROSS (
> http://techbase.kde.org/Development/Tutorials/Kross/Introduction), and
> more. But I had never any reason to do it.
>
> As a dummy solution we can add a quick workaround with some user-defined
> menu entries which can run your scripts with some additional arguments like:
> callExternalScript("python foo.py /path/to/databasefile.db providedusername
> pass...");
> and handle its results
>
> > This is something that could be really useful for me and I would do the
> programming myself. However, I only know python programming and have some
> basics of qt. I would like to know if someone could be interested in helping
> me. I can also do the programming myself, but I need an introduction to
> Sqliteman source code and eventually some tips to learn fast C++.
>
> Coding in Qt4 is quite easy. It handles most of memory management for you
> itself like Python does. I can say there is only syntax different between
> PyQt and pure Qt4 if you will make only simple things like forms, image
> loadings, or R interpret calling. Please take it as a "motivation and
> introduction" only ;)
>
> ad Sqliteman code internals: most of the code is (I hope) self explanatory
> - there are no low-level techniques used. Most code files contain "extended
> functionality" like various dialogs (new db objects, populator, exports,
> imports, ...) - it's usually small file with little "blackboxed"
> functionality.
>
> most of code should be documented, 1.2.x docs are available online:
> http://docs.sqliteman.com/devel-doc/annotated.html
>
> main app is located in litemanwindow.* which connects basic GUI together.
>
> Direct data manipulation (without sql) is done in dataviewer.* and its
> sub-components.
>
> database.* files contains functions to query or manipulate raw SQL
>
> Personally I'm not happy with current architecture of DB transaction
> handling as it mixes raw SQL access (BEGIN; foo; COMMIT;) with Qt4 QtSql
> module transactions (in dataviewer). I had plans to write own driver to be
> used in the app as only db access method one day but there is no motivation
> for now on my side.
> And of course - all DB access is done in main app thread so the app can
> looks "frozen" in some circumstances.
>
> Anyway, I think it's enough for quickstart. Feel free to ask.
>
> cheers,
> petr
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
>
> iQEcBAEBAgAGBQJNJEwIAAoJEC8yRjM4uE2tNvUH/1fdYSeNP3Sg+Qe5yGlNXzB5
> wSYnICpxBT6vCKBsonF1QGPXDFo6z3YWQNCSy23KN+9ODCNDyYnEHBVawysGHpO3
> Hw8fRiAP81o7DTiFPdhke3YT6TnE6z4TwNaQjyOM4/Dah3kFzvxgfOpWg72HmDmJ
> SHTUgArvZ6BmlYbIkWmvgbMbJNHhKwVSuxRXOxxQT8Tdilih0nCQ9VTt6E5hT0Bc
> 0pX0mfUWHXPlV3IytpEIfNwSqnqzjULZMXALxpOOWGxm3d0utgWugKpPTeFT6NhO
> JNvlClgwOpEtCCpSpJo+nKl2kEqadaNyZK/rWohJajkCoXPY2QrDqwyq70zyy4I=
> =pxcD
> -----END PGP SIGNATURE-----
>
>
> ------------------------------------------------------------------------------
> Learn how Oracle Real Application Clusters (RAC) One Node allows customers
> to consolidate database storage, standardize their database environment,
> and,
> should the need arise, upgrade to a full multi-node Oracle RAC database
> without downtime or disruption
> http://p.sf.net/sfu/oracle-sfdevnl
> _______________________________________________
> Sqliteman-devel mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqliteman-devel
>

Thanks for answering.

SCRIPTING
KROSS looks quite interesting. From my understanding, that could be coded
with little modification on the Sqliteman actual source code, this is
interesting as it separates this part from the Sqliteman development.
I did not quite understand the dummy solution. Do you think this is fast to
implement?
Anyway, I will have to see if I am able to implement such a thing on my own.
Probably I can, but it will take some time as I have to start from 0 with
C++ and compiling.

FUTURE DEVELOPMENT
I don't know if I was the missing element. What I know is that with a
pluggable solution things get interesting. There can be graphs (matplotlib),
statistics (SciPy; R) and dynamic report generation.
My best help could come from Python and R scripting. I have experience in
this and I think I could program a few plug ins for start.
Maybe we could implement a beta for scripting just to see if this is of some
interests for end users.

By the way, I will ask some help to a friend of mine who knows some C++.
What you guys think in general by the way?

On Wed, Jan 5, 2011 at 11:46 AM, Petr Vanek <pe...@sc...> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> hi Alessio,
>
> thanks for you interest.
>
> The situation around Sqliteman is not good. There is no progress in it for
> months. The 1.2.x versions are used still, but the development 1.3.x never
> touched its testing releases although it contains many improvements.
> The main reason is that I don't have free time to work on this software and
> mainly - I don't need it to my regular work too as there is no sqlite db
> around me now.
>
> There were some movements how to start its development again but with no
> (or minor) success.
>
> Maybe you will be the missing startup element? Who knows...
>
> responses to your questions follows:
>
> On Jan 1, 2011 (Saturday), at 5:03 PM, Alessio Civitillo wrote:
> > I have been using Sqliteman for quite a while and it is indeed the best
> Sqlite manager out there. Lately, I have been thinking of implementing
> Sqliteman with an "analytic" part. I have been using Sqlite to store data
> for my analysis and I found it really good. My main issue is that I use
> python and R coding to make my analysis and I have scripts all around. My
> thought is: what if Sqliteman could be implemented with a pluggable analytic
> part? So, maybe with the possibility to create ad hoc analysis plug ins with
> python.
>
> There is no support for 3rd party plugins.
> There is no scripting support too.
> There is support for sqlite3 extensions in 1.3.x.
>
> There are some solution how to enable scripting in Qt4 app - manual
> bindings, KROSS (
> http://techbase.kde.org/Development/Tutorials/Kross/Introduction), and
> more. But I had never any reason to do it.
>
> As a dummy solution we can add a quick workaround with some user-defined
> menu entries which can run your scripts with some additional arguments like:
> callExternalScript("python foo.py /path/to/databasefile.db providedusername
> pass...");
> and handle its results
>
> > This is something that could be really useful for me and I would do the
> programming myself. However, I only know python programming and have some
> basics of qt. I would like to know if someone could be interested in helping
> me. I can also do the programming myself, but I need an introduction to
> Sqliteman source code and eventually some tips to learn fast C++.
>
> Coding in Qt4 is quite easy. It handles most of memory management for you
> itself like Python does. I can say there is only syntax different between
> PyQt and pure Qt4 if you will make only simple things like forms, image
> loadings, or R interpret calling. Please take it as a "motivation and
> introduction" only ;)
>
> ad Sqliteman code internals: most of the code is (I hope) self explanatory
> - there are no low-level techniques used. Most code files contain "extended
> functionality" like various dialogs (new db objects, populator, exports,
> imports, ...) - it's usually small file with little "blackboxed"
> functionality.
>
> most of code should be documented, 1.2.x docs are available online:
> http://docs.sqliteman.com/devel-doc/annotated.html
>
> main app is located in litemanwindow.* which connects basic GUI together.
>
> Direct data manipulation (without sql) is done in dataviewer.* and its
> sub-components.
>
> database.* files contains functions to query or manipulate raw SQL
>
> Personally I'm not happy with current architecture of DB transaction
> handling as it mixes raw SQL access (BEGIN; foo; COMMIT;) with Qt4 QtSql
> module transactions (in dataviewer). I had plans to write own driver to be
> used in the app as only db access method one day but there is no motivation
> for now on my side.
> And of course - all DB access is done in main app thread so the app can
> looks "frozen" in some circumstances.
>
> Anyway, I think it's enough for quickstart. Feel free to ask.
>
> cheers,
> petr
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (Darwin)
>
> iQEcBAEBAgAGBQJNJEwIAAoJEC8yRjM4uE2tNvUH/1fdYSeNP3Sg+Qe5yGlNXzB5
> wSYnICpxBT6vCKBsonF1QGPXDFo6z3YWQNCSy23KN+9ODCNDyYnEHBVawysGHpO3
> Hw8fRiAP81o7DTiFPdhke3YT6TnE6z4TwNaQjyOM4/Dah3kFzvxgfOpWg72HmDmJ
> SHTUgArvZ6BmlYbIkWmvgbMbJNHhKwVSuxRXOxxQT8Tdilih0nCQ9VTt6E5hT0Bc
> 0pX0mfUWHXPlV3IytpEIfNwSqnqzjULZMXALxpOOWGxm3d0utgWugKpPTeFT6NhO
> JNvlClgwOpEtCCpSpJo+nKl2kEqadaNyZK/rWohJajkCoXPY2QrDqwyq70zyy4I=
> =pxcD
> -----END PGP SIGNATURE-----
>



-- 
Regards,
------------------------------------
Alessio Civitillo
ale...@gm...
Mobile: (0045) 52645608
Linkedin: http://it.linkedin.com/in/alessiocivitillo