I can hardly believe it's been almost a year since my last blog post. It's been a fairly busy year overall. I am now a full gentoo developer, I've taken on more programming tasks and projects...

  New Projects:
- PyGPG  Is a python interface library that runs gnupg in a subprocess while mining all data that it exports through it's status-fd output. It aims to be a complete, flexible solution to using gnugp via python. It is early in it's development, but there is an -9999 ebuild in the tree.

 - Gentoo-keys   Is a gentoo gpg key manager app that will use pyGPG to download the binary gpg keys from key servers and manage the gentoo release and developer keys that will be used by layman and other apps to validate the gpg-signed repositories.xml list, the install media, etc..  I may also use it to manage any overlay gpg-keys that may become available in the future. So far I have the main seed file management routines in place. Next I will begin coding the routines needed to manage the downloading, removing, moving of the gpg keys to the correct keyrings and validate it matches the fingerprint from the seed file. Once that is done, I will likely do some coding to make validation using those keys easier, along with code for the git commit validation hooks which are needed for the tree migration from CVS to Git.

-  Mirrorselect  I've just done an update for the make.conf move from /etc/ to /etc/portage/. From there I proceeded to convert it from a nearly monolithic script into a full python lib with a complete api. That will leave it open for creation of a gui or inclusion of it's capabilities into other applications. I also did some python 3 compatibility fixes too.  It has just been released as app-portage/mirrorselect-2.2.0-r1.

  On-Going Projects:
- Layman   I have finally gotten around to fixing the last of the bugs in 2.0.0's code and re-released rc5 as 2.0.0 final. Rc5 has been out for a month now without a single bug filed against it. So look forward to having 2.0.0-r1 stabilized soon. Next on layman's agenda is complete the gpg signed repositories.xml support.  For which working code is already in place in the -9999 live ebuild. But so far infra does not have a regular gpg-signed list available. There is only a testing one that is not updated since it's creation. I have 2 others on my dev space as well, but they too are not updated so not usable for use other than testing. After that I plan to continue the python 3 code migration.

 - Gentoolkit  There are a number of small bugs that need attention along with some enhancements and updates needed.

 - Porthole  I've been neglecting it for far too long. It is getting behind on many changes to portage and pkgcore. Plus I have several improvements half finnished.

 - CoreBuilder  Like porthole, I've not done any work on it for quite some time. It is a web based interface to package management similar to porthole and uses portholes new backend structure. It's primary reason for being is for tinderbox testing of packages.

 - Esearch  Nothing new to report after we finished the major update to it's code. There have been a few minor bug fixes and is very stable. It is also python 3 compatible and it's db can be loaded, updated with either python 2 or 3 and fully usable by the other.

 - Portage  After several years, the emaint re-write I did has been merged into portage and people have been using it along with a few new modules that I've coded along the way. I have an idea in the wings for a module to assist users in cleaning up any "-MERGING-" pkgs which have failed during the move from the build sandbox to the live file system. Currently they have to be dealt with manually.

 - Pkgcore  I need to get working on the backend manager code for porthole and coreBuilder. From that there is likely to be some enhancements to pkgcore's api.

 - Gpytage  While I've not been doing any coding on it, Ken69267 has returned from the brink of gentoo retirement and is doing some work on it again. We look forward to the changes and enhancements.

 - Etc-proposals  Nothing new to report, it's code has been working fine.  But if anyone wants to work on updating it's QT/KDE interface, that would be welcome to all it's QT/KDE users.

Well, Christmas decorations, my wife and kids are waiting for assistance...

Later everyone :)

Lazy blogger

Well, I've certainly been a lazy blogger...

Short story...

Porthole
and Cazou (student) were accepted as a GSOC 2011 project. Unfortunately, it failed by midterm :( Still, there was a decent start to the backend re-write and the code submitted was good, so provided I can get my butt in gear, I should be able to complete it. Once that is done, it will be easier to maintain porthole. Currently, it is in an in between state. I mean that in both the front-end and the back-end. I began making changes to the front end to be able to add a proper layman plug-in that can use the category & package view panes. But that code is not completed and needs more testing, debugging. Some of the other things need some updates as well to keep up with changes in portage's code and config files.

Layman
I've gotten the 2.0 api nearly completely debugged. I have just a few more small changes to make to release rc4 witch will be the last rc and should end up as the final 2.0.0 release. More about that later, I promise. Also I made a request to the infrastructure team and antarus stepped up and produced some great stats for the last half of 2011. The new http headers it is sending with the requests for an updated overlays list has resulted in a better than a 20:1 ratio of 304 errors (list not updated since last time) to sending a new updated list. :D more on that to come...

Other news
As of Nov. 19, 2011 I am now a gentoo staffer. I've also just recently completed the ebuild quiz, so will become a full dev once that is reviewed and accepted. So, once that is finalized I should have less excuses, more incentive to get some of these lingering task completed.

Gotta go for now.

Layman-2.0 development testing

As reported in some earlier blog posts, I've recently taken over maintaining and developing layman. I have been working on replacing the high level api with one better suited for consumer apps use. I've also changed the cli to use the new api and that has resulted in about a 20-35% speed increase for the testing I've done.

I've also been extending the mid level api a little so that it is more flexible for use as a lib in other applications. It does not require an overlay definition be input via xml file only. It can now accept a python dictionary to define an overlay or for that matter any VCS be it a development checkout, etc.., also it's location can be anywhere in the filesytem. That is something portage/pkgcore or a layman gui might use, it is not intended for the cli interface.

Some notable changes:

* New high level API
* New cli connection to the new API
* New add, sync, postsync config options for each repo type
* repo xml definitions can now be placed in the /etc/layman/overlays/ dir and they will automatically be added to the available list without adding them to the config manually
* New multiple config functions/classes allows for more flexible API use when using layman as a lib.
* It now timestamps the remote list and only downloads a new list if there is a change since it was last downloaded.
* New repo type g-common. The dep is not yet included in the ebuild as it is supplied via app-portage/g-cran from the science overlay. But if you use g-cran packages you probably already know this.
* New Message class fully capable of re-directing output to the defined output passed in. Again mostly for guis or consumer apps use.
* other code cleanups, small improvements.

So those that would like to help test:

Code:

echo 'app-portage/layman **' >> /etc/portage/package.keywords # emerge =layman-9999

Please report any issues you have with it so I can get them fixed
I would like to try and stabilize the code for a release sometime in the near future.

gentoo forum thread

Porthole 2011 GSOC project starting

Porthole has been chosen as one of this years GSOC projects. The project will encompass a complete re-structuring of porthole's backend so that it can make use of a new (soon to be stable) portage public API. Stable at this point means mostly a non changing interface for directly operating portage, getting information directly from portage while the backend code can change without fear of constant breakage for consumer apps like porthole.

The student doing the work is Cazou, the same person that worked to port kuroo to QT4 last year. This restructure is needed so that porthole can work even better with portage. It is also needed to be able to work properly with pkgcore which a backend for it will also be produced. That will give you more options as far as package managers are concerned.

More on this later...

All is not lost

Hey everyone. No I've not dropped off the edge of the earth.

I am gearing up for another season of mentoring for another gentoo gsoc project. This time it will be for porthole. This project is to finish to back-end re-structuring and create a new pkgcore back-end.

Other news, I am now the developer/maintainer for layman. I have implemented the new api I did last year. Extending and improving it some more. I have also moved the cli to use the new api. This has resulted in a 20-35% speed increase for the testing I've done. I've also started a porthole plug-in for it.

Even more porthole news:

Paul Varner has done some work and has helped me get porthole working in a prefix install. I am finishing up the changes, porting them from the dev branch into the master branch code. I'll push out a live ebuild and/or a -0.6.2_beta release hopefully soon. There has not been much in feature additions. mostly more code re-structure and cleanup so it should be easier to modify and maintain. some bugfixes...

Here is a screenshot of porthole running on fuzzyray's mac.

layman

Well, I've gotten everything rebased to layman again and merged into master. Along the way are several more fixes to the work I started in overlord. It is available in the layman-9999 ebuild in the main portage tree.

I'd appreciate all those that test it to make sure I haven't broken anything. I want to make a few more smaller changes to the code before making a layman-1.5 release. So test away, let me know of any problems or quirks that need tweaking.

With that part behind me , I've started to work on extending porthole's plugin system to be able to add custom views via plug-ins. My main goal is to get a layman plug-in working for porthole. I plan to use the existing porthole layout using the category listings for the installed, main remote list, add in lists. The package view for the short info listing and the summary (or possibly it's own tab) for the detailed info. With that part accomplished, I'll add the ability to add, delete, sync repositories.

Back to layman, I was thinking of adding a file in the same vein as portage and the main tree to record the last sync time. With that, i'll extend the layman cli to include an option to list the last sync time(s). It will also be available to api consumer apps like portholes layman plugin.

Layman related, I've been in touch with the dev that created a new git-2.eclass to see about extending it to be able to handle multiple listed EGIT_URI's with in it's declaration. That way if there are multiple protocols available or server locations it can be cloned from, it can cycle through them until one succeeds. just like layman does for adding an overlay to your system. That way if you are behind a firewall and the git:// protocol can not get through, it will try the next uri which may very well be an http:// protocol which can succeed. That should make live ebuilds more accessible to users.

meanwhile back to coding... :)

Getting Busy :)

Well, I know i haven't been very chatty here on my blog. Things have been getting quite busy.

I've been getting some work done on porthole:

1. I've split up the mainwindow code into smaller managable chunks that all stack together.
2. I've fixed the multi-selection for all commands like emerge, unmerge, update. Unfortunately I discovered a bad side effect of that. While selecting a bunch of dependencies I wanted to upgrade. I came across a pkg I emerged with --oneshot that I didn't want anymore. So clicked unmerge (old behavior would just unmerge the pkg selected), then continued selecting and then upgrade. To my surprise I discovered everything selected to upgrade failed due to wget failing. Then I discovered all the unmerge's which crippled my system. Fortunately I have --buildpkg enabled so was able to emerge a number of pkgs from the backups. My system is happily merging updates again. I still need to run revdep-rebuild after to catch any more missing deps from my unmerge stint.

I got to talking with Wizzleby (kportagetray developer) about the layman api I coded awhile back. That got us both going on some preliminary designs for collaborating on both a gtk and kde front-end to layman. Since I had not been getting any feedback about the new api from the main layman dev. I decided to start a fork to work on it more so that it can be released and ready for the guis front-ends. Well that lasted about a week, I made good progress on it, replaced the cli interface to use my new high level api. That resulted in a 20-35% speed increase. I am now the primary layman developer and maintainer. So now I have to port my changes back to layman's namespace. Anyway the new project's name is Overlord. I will keep it for the guis front-ends and possibly a few extras for layman. So give me some time and I'll have the changes in the Layman-9999 live ebuild ready for more testing.

I have been pushing to get the gentoo-guis mail list back in action again and discovered it hadn't been de-activated after all, but it was not listed either. So any of you interested in joining, it is gentoo-guis+subscribe@lists.gentoo.org. We welcome all

to join in.

We have started a new gentoo-guis overlay for our live ebuilds and development work. It is not yet listed in the main repositories list, but I do have a gentoo-guis repositories.xml file in the overlay you can download and configure layman to use to add it. Actually I made that easier to do in overlord. Those repo.xml definitions can be plugged in for automatic detection and inclusion. Yes, that will be ported to layman too :)

So, I am accepting volunteers for help with things on porthole, layman, overlord,...

:D