Busy, busy, busy,...

Wow, April has been a very busy month. I meant to keep you updated to the goings on, but didn't seem to get to it. My daughter's softball season has been up and going, not to mention all the other activities that she needs to be brought to, etc.... I have been starting back to work also on a part time basis, but my knee doesn't seem to be handling more than a 4 hour shift. I have also become a gentoo mentor for the Google Summer Of Code program. That has taken up quite a bit of time as well reading/evaluating proposals. I believe the announcement for the GSOC will come out later today (this is still my night before). I have also been contacting the other portage gui front-end developers to begin discussing the upcoming merge into portage of a public API. Coding wise I have not been able to accomplish much, mostly fix some bugs in gentoolkit, and do a little more work to portage's emaint that I've been re-working so that all its modules are available as import-able modules for use by portage, other maintenance tools, or even possible gui front-ends or plug-ins.

Oh, and some great news!!!! I have put in a stabilization request for porthole. So far it has a ppc stable keyword, approval for x86 and sparc to go stable, I just need to get the ebuild updated. So if you use porthole on any of the arch's that aren't marked stable, get after the arch team to stabilize it. If you are using it on an arch that is not yet listed in the ebuild, speak up, open a bug to get your arch added to the list.

Anyway, I have a busy day ahead for me tomorrow, so I should head off to bed. I'll be in touch about the gsoc project I'll be mentoring after the official announcement has been made.

status update

I've been a little busy lately and haven't had much time for a new post. I have the eclean re-write working good and have several unit tests for it, but to test it all would mean having 10 times the code just to run the tests on the real code. You will be able to try out the re-write in the gentookit-0.3.0_rc10 release coming out soon. For cleaning the packages directory it now relies on emaint from portage to fix the Packages index file. For now it runs it in a sub-process from eclean, but I hope to get some changes approved to emaint, so that eclean will be able to use it directly. It will then not disrupt the output from eclean like it will for now. Gentoolkit is now going to be python-3 capable and will require a minimum of python-2.6 to work correctly. Also, for those not following the gentoo-soc mail list, an idea for a network enabled eclean version was brought up that would be useful for central NFS servers serving the distfiles or packages directories. With that i have been able to deduce that it will not take much more work to add shared disfiles and/or packages ecleaning ability. So perhaps look for that feature in the near future, perhaps in gentoolkit-0.3.1.



It also looks like my knee injury is nearly healed up and I may be returning to work soon. A good thing about that is it might help to shed the 20+lbs of weight I put on since the injury.

I also need to code some additional modules for the new analyse utility I started, along with a major backend re-structuring for porthole. I've been also asked to clarify some of the analyse descriptions, abilities that I posted about earlier. So look for that this week. That's it for now.

I finally got another porthole release out. I had been dragging my heels a little about it, but I also found a fixed a few more bugs. So this should hopefully be a release that makes it to a stable keyword. There was one potential bug in 0.6.0 that I found which could make an annoyance to fix for a user, not critical. So I did not push for it to go stable. (Although far buggier apps get stable keywords)

Porhole-0.6.1 has added a number of small features, like a highlighted eduild viewer which uses gtksourceview and the gentoo syntax highlighting rules. The Changelog has been completely re-vamped and now sports a new custom highlighting view. I so need to improve it slightly for dealing with overlays that do not have a changelog file so that it will look in the main tree for that one.

That covers most of the visible changes to porthole since the 0.6.0 release. I have been busy working on elclean code and have it ready for adding in an emaint binhost call to fix the packages index file. So on that note with the porthole release out the door I got busy splitting up the portage emaint program so that it's modules could be used directly by other python apps or portage front-ends (think porthole plug-in). So I'm hoping the powers that be like the emaint changes I've made, one of which is a modular plug-in system for emaint modules. It makes it easy to add or remove modules to the utility as changes are needed or additional functionality is created. Without the need to change the cli or gui front-end. So with that pretty much done I'm getting back to completing the eclean rewrite so it can be merged back into trunk for a release.

I have also done some more work on the new analyse utility, but will leave that for another post so I can better explain it.

New gentoolkit utility --- analyse

Well, I've gone and done it again, I haven't finished the eclean re-write and I've found something else to work on. It all started from a forum post. He wanted to fix his mistakes and it looked like the only way was the hard way.

Having been doing more gentoolkit coding I put together a quick and dirty python script to output some USE flag info about his installed pkgs. With that and me saying I could add some more code to spit out a package.use file populated with what is installed/what flags enabled/disabled. I have now been coding it up and have it now working for USE flag analysis and rebuild. I plan on adding more so it can analyse and repair keywords and mask files too.

I also would like to code up yet another /etc/portage/package.* cleaning tool. Since It'll have nearly/all the functions needed for gathering the info, may as well do that too. All the main functionality will also be importable for other apps to use as library functions/methods.

analyse rebuild -pv use

analyse analyse -uv use

left: showing the output of the report. Enabled USE flags are marked with a leading + and colored blue. Disabled USE flags are marked - and colored red. Unset USE flags are plain text.

It also indicates if the flag is a system default, the number of packages that have it enabled/disabled/unset and using the --verbose option lists the packages.

right: Part way down the report. notice that mplayer has faac faad flags enabled but ffmpeg does not. Something that I didn't notice before. Now I know to set them for ffmpeg too.

This last one shows the totals at the end of the analyse report.

Also if you can think of a good "e" name for it to go along with the other gentoolkit apps like eclean & equery. Post a comment.

some of the names so far:

• erepair
• esylana (analyse spelled backward)

Distractions

I've been distracted away form porthole code lately. Well not completely, I do have an ulterior motive. I am working on gentoolkit code again. Namely a modular rewrite of eclean.

eclean was an all in one python script that contained a number of unwieldy methods and functions which made it difficult to modify and nearly impossible to create test functions for. One of the primary reasons I started on it was to split the back-end functioning code from the output printing code similar to most of the newly redone equery modules. That way the code can be used by other applications. So, I began by separating the script into a python package containing 5 files and a nearly empty script that just loads and calls the main() .

Those 5 files being:

1. cli.py to handle all command line interface argument parsing and start the requested action.
2. search.py with all the search related methods and functions which determine the files to be cleaned.
3. exclude.py containing the exclusion file parsing and handling code.
4. output.py for the screen output and user interaction.
5. clean.py to handle the actual cleaning process.

The end result should be one that has a number of automatic tests run on it to check that any code modifications are performing correctly as well as code that will be easier to maintain.

It will also then be possible for the code to be used in other apps such as a planned plug-in module for porthole. :)

gentoolkit fun

I've been having some fun working with djanderson on the gentoolkit equery code. We've made great speed improvements to the depends module for reverse dep searches. The real speed improvements were by djanderson's rewrite of the base search code after I had initially split it up into separate search and print modules. There have been 2 benefits so far

1. Is that consumer apps will be able to import the python modules for its own use without having to parse terminal output. Great for making a plugin for porthole (my original intent)
   
2. Is the major speed improvement as a result of the complete rewrite of all the dependency related code from the original package object into its own class. The speeds are now about 25% that of the original code. most equery depends package will now be around the 2 to 3 second mark instead of 8 to 12 seconds. Really long detailed searches are down from 2 - 3 minutes to around 2o seconds!

With the depends module nearing its completion, I'll get back to working on the belongs module. Maybe we'll be able to get more speed out of that one as well.

Back to coding

Current coding work I'm poking at:

Porthole

• I've just added syntax highlighting to the ebuild view courtesy of portato code. I would have done it earlier, but for trying to get the final version done with. I didn't want to add yet another change that might delay it's release due to more testing being needed...
• I've updated the file chooser widget used from the deprecated gtk.FileSelector to the new gtl.FileChooserDialog. It does also have file filtering options, and if I've done my testing correctly should be working properly everywhere it's used.
• I've gotten back to the dev branch that I'm using to split out all hard-coded emerge commands. The commands used to build the command lines used for emerging, etc. will come from config file(s). That will allow a user to customize the actual commands sent to the terminal. ie. use a custom wrapper script. It will also make it possible to use pkgcore commands for the merge. Once it is working, I'm sure some pkgcore devs will help to make a pkgcore backend for the information gathering as well.
• I've added a USE Flag tab to the package notebook. It is currently not selectable since I do not have any code for it yet. I plan to separate out the advanced emerge use flag portion of code into its own module so it can be used in both places, or anywhere else in the future.

Gentoolkit: I've begun poking around the gentoolkit code to see about using it for a plug-in module. I'm sad to say it was not designed to be used as an imported module. That said, I've seen on #gentoo-portage that djanderson has been working on the code. I've contacted him and have begun re-writing some of the code to separate out the data gathering and printing to the screen. This will improve the code base overall along with some major updates he is working on which will make the code py3k compliant as well as keep it up to date with current portage.
Together the changes should speed things up some for command line users. It will also make it possible for apps like porthole to import the gentoolkit code for its internal use and display of info such as reverse dependencies. (the first module I worked on splitting up)

Gpytage: A nice little gui utulity program for editing your /etc/portage/package.* files. I had helped Ken out a little in the past. I have started to poke at him some more to finish some updates he was doing so I can help debug it. I also hope to add some code to validate the entries and report any obsolete data & package entries. That code will come from another standalone utility script that I came across some time back. It will need some slight re-write to make into a library module and separate command line script. I am getting anxious waiting to make a plug-in module for porthole which will be linked in to the summary view and its menus. The existing version in the tree is somewhat outdated with portage, so will not be use-able. The new code will have a significantly different API.

IRC: I've created a #porthole support channel on freenode. feel free to drop by and chat. I can also be found on #gpytage, #gentoo-guis, #gentoo-portage, #gentoo-amd64, #gentoo-desktop, #pkgcore. I have been playing around with bots a little as well. You'll find a basic bot called portholes there. I intend to replace "portholes" with a different more powerful bot when I get the chance to get it set up.