Silicon Island

This blog has moved to Wordpress.com

I've moved my blog, you can read my new posts here https://siliconislandblog.wordpress.com/

Despicable SPI

So apparently, this happened, and then this. Long story short, the elementary OS guys had been offered to use SPI as the legal entity to represent the project, something they didn't need at all, and since they didn't, Joshua Drake, apparently a director at SPI, decided to threat them with bad press all over if they didn't agree to join SPI. Which he then did, he started several threads on reddit and wrote a blog post trying to undermine the project, the post is now deleted, and this aberration of an apology (which is total BS and shows how much of an ass he is).

I seriously don't get why this guy has not been fired from the SPI organization immediately, this sort of bullying behaviour should not be allowed and, at least in my book, an apology means nothing. Someone like that does not belong to an organization that is supposed to help free software thrive and protect its communities.

I don't get how SPI expects the community to trust them at all after this.

I am really angry at this and I would like to express the elementary OS guys all my support.

Re: Scammers at promo-newa.com

Beware of promo-newa.com, they scammed my colleague and friend Richard. As someone with experience in importing from China I know how scary and risky it can be so I completely sympathize with them. Apparently they sent hacked 96Mb flash drives that reported to be 1Gb flash drives.

Let's make sure the internet is filled with references to this scam. Also, if you live in Shenzhen and/or can think of any of helping him that'd be really nice.

At the very least, make sure you share this post around in your preferred social media wall!

DevX Hackfest 2015

Yesterday I arrived to Cambridge to attend the DevX hackfest. Loads of good stuff going on, I am mostly focusing on trying to integrate the hundreds of ignored pull requests we're getting in Github's mirror with Bugzilla automatically. In the meantime loads of interesting discussions about sandboxing, Builder, docs and mallard balls being thrown all over the place and hitting my face (thanks Kat).

It is real nice to catch up with everyone, we went for dinner to a pretty good Korean place, I should thank Codethink for kindly sponsoring the dinner. Afterwards we went to The Eagle pub, apparently the place where DNA discovery was celebrated and discussed.

And this morning we are celebrating Christian Hergert making it to the 50K stretch goal just before the end of the crowdfunding campaign for GNOME Builder.

I would like to thank my employer, Red Hat, for sponsoring my trip here too.

Back to the kingdom

And not the kingdom of Spain unfortunately (unfortunately because I miss it and because it's still a kingdom). In a few months (not sure about specific dates yet, probably in early 2015) I will be moving back to the United Kingdom, this time to the larger metropolis, London. Don't panic, I will still be with Red Hat, there won't be a lot of changes in that front. In the meantime I will settle back in Gran Canaria and will be flying back and forth on a monthly basis.

I must note that when I made the decision to move to Czech my plan was: "I do not have a plan", just enjoying it and trying to make the best of it without thinking in deadlines as to when to move back to Spain. Red Hat has been a very welcoming company in which I feel just like home and Brno has been a very welcoming city and this is definitively a part of Europe that is worth experiencing. I've met terrific people during this period both inside and outside Red Hat.

There was, however, a little problem.

Something altered the mid-term plans, a few months before I moved, when the decision was already made, I met someone very special with whom now I want to share my life with. After 16 months of  carrying a distant relationship it was due time to find a place where we could be together, after months of planning and considering options, London presented itself as the spot to make the move as she found a pretty good job there.

While I am going to miss sharing the office on a daily basis with awesome people, I am looking forward to this new chapter in my life.



I want to note that I am deeply thankful to Christian Schaller for his tremendous amount of support during my stay in Brno and for working with me in figuring ways to balance my professional and personal life. I also wish him the best of luck with his new life in Westford, I'm certainly going to miss him.

On the other hand I guess this means I'll show up at the GNOME Beers in London more often :-)

On OPW, leadership and PGO

Recently Philip decided it was time to call for some attention.I happen to agree with him on the need to focus on developer experience, that's why I organized the first hackfest on this topic last year and attended this year. There are plenty of conversations around this and Philip, if you care so much, maybe you could attend or help, there's a lot to do and so few hands.

I've been asked to remove your blog by several people and I've reached the conclusion that it would be a really bad idea because it would set the wrong precedence and it would shift the discussion to the wrong topic (censorship yadda yadda). Questioning OPW should be allowed. The problem with your post is that if not questioned by other people (as many have done already) it would send the wrong message to the public and prospect GSoC, OPW and general contributors. Your blog was the wrong place to question and your wording makes it clear that you have misunderstandings about how the community works.

You want to make things better? Why don't you start by learning how to work with others and contributing yourself? You think we need better leadership? Why don't you learn what it takes to become a leader? (hint: your blog post doesn't help)

Perhaps your lack of contact with the overall project and your abscence from most events makes you not realize how possitive OPW has been, OPW has been a lot more successful than GSoC in retaining contributors and bringing diversity to our contributor base (and I don't mean gender diversity, but diversity in the nature of those contributions). I happen to have a pretty good picture of this because I get to manage the blogs of the people who stay and the people who leave. Without OPW GNOME would be worse community wise and project wise and this is not an opinion, it is a _hard data_ backed fact (other posts have enumerated the contributions that would have not happened otherwise so I will not do that here).

There are plenty of questions that I think are healthy to ask: for how long do we do OPW? Is its success only due to it being targetted to women or is it successful for something else? You should have a conversation with Marina and other people involved with OPW and gather an understanding before making assumptions and throwing assertions. And you should respect what people chooses to do within the project, it's their goddamn time after all. In open source no one gets to dictate what nobody does (though alignment is always good if can be achieved), people work in what they think its important and they try to do it together.

I think you should also watch this video, it might give you some understanding on why GNOME is as responsible for equality as any other entity.

DevX hackfest

Two weeks ago I flew to Berlin for the 2nd GNOME Developer Experience hackfest.

The event was quite productive and allowed us to move things forward in different fronts. There has been many blog posts about most of the on goings so I'll stick to what I did.

I joined the discussions about the data model APIs with Ryan and Lars and tried (and failed) to include some new widgets in Glade (GtkStack). Turns out it's not as simple as I thought and that there was already a patch in the works for this so I gave that bullet up.

Later on I sat down with Tomeu to try to revive my former attempts to revamp the API reference documentation UI a bit. He managed to produce a set of JSON files I could use to produce the UI from. My intention is to create a web frontend that works mostly client-side and is off-line friendly (pre fetching all the content) for later use. This could potentially enable a DevHelp replacement through a Web (Epiphany) App.

I have some code working using jQuery, I would like to eventually switch to AngularJS as it has built-in templating support, currently I am generating quite a bit of HTML myself and that would eventually become a maintenance/styling nightmare.

Adventures in Go land: high performance custom HTTP(S) service

The problem

While working on a side project I was confronted with the problem of serving a JSON object to potentially tens of thousands of users representing the state of the app (think you are following the results of an election live and you want data to be updated every second).

The web service is really simple, it grabs the payload from the data store (disk/Redis/Mongo), uploads it every second, and serves the content accordingly.

Initially I implemented this using Python and Flask, I must admit that I didn't try too hard to get the data in an async fashion since it wasn't obvious how to do this with Flask. Using twisted could have been a better choice.

Initially I attempted to use Varnish as a front-line HTTP cache with promising results, however using HTTPS became a requirement pretty soon (due to CORS policies) so I had to discard Varnish rather quickly.

Go's bare HTTP server performance

 

However I took the opportuninty to teach myself Go which I knew it had pretty good support for concurrent operations and that the Google guys already have a success story serving static data with it. Boy have I been surprised.

Take the simplest form of an HTTP server that serves a common JSON payload for all the requests:

Gist link

Note that it reads the data just once from a json file in the disk. No updates every second, for the sake of showing how well Go can do as a static HTTP server, these are the performance results using httpress  on my ThinkPad T430s (i7-3520M 2.90GHz/8GB of RAM):

$ ./httpress -n 100000 -c 1000 -t 6 -k http://localhost:8080
TOTALS:  1000 connect, 100000 requests, 100000 success, 0 fail, 1000 (1000) real concurrency
TRAFFIC: 4195 avg bytes, 109 avg overhead, 419532658 bytes, 10900000 overhead
TIMING:  3.752 seconds, 26646 rps, 112007 kbps, 37.5 ms avg req time

That's 26646 request per second with 1000 concurrent keep-alive connections in 6 parallel threads without doing anything special, I must say that I'm impressed that the Go developers have put so much effort in their built-in HTTP server implementation.

Go's HTTPS performance

 

Now that Go started to look as a viable option, let's see how it behaves as a HTTPS server. Here's the link to the modifications to turn the previous example into an HTTPS service, note that you have to generate the certificates.

TOTALS:  1000 connect, 100000 requests, 99862 success, 138 fail, 862 (862) real concurrency
TRAFFIC: 3994 avg bytes, 109 avg overhead, 398848828 bytes, 10884958 overhead
TIMING:  22.621 seconds, 4414 rps, 17688 kbps, 226.5 ms avg req time

Damn you secure webs! That's a big downgrade in performance... 4k rps now. This made me realize that this new security scheme for cross site content implemented in browsers is going to kill a lot of trees. At this point I wanted to investigate further for possibilities boost this up.

Revisiting the front-line cache idea: Cherokee

My suspicion is that the TLS handling inside Go's net/http has some chances to be improved, but I didn't want to get my hands dirty there, so I started to think about off-the-shelf alternatives.

I stumbled upon Cherokee's front-line cache by reverse HTTP feature which is really easy to setup thanks to the cherokee-admin interface. This means that I run my original plain HTTP Go implementation as-is behind Cherokee so that it handles the TLS by itself.
These are the results:

TOTALS:  1523 connect, 100000 requests, 99511 success, 489 fail, 1000 (981) real concurrency
TRAFFIC: 8635 avg bytes, 142 avg overhead, 859277485 bytes, 14131208 overhead
TIMING:  12.975 seconds, 7669 rps, 65735 kbps, 130.4 ms avg req time

There's quite an improvement in latency, though the requests per second improvement while worth the while is not as big as I would have expected.

Conclusions

And these are my findings so far, I want to try to configure Cherokee so that it only talks to the backend server once every second and cache the requests that way, haven't figured that out yet.

As per my experience with Go, it has been mostly pleasant, it's nice to have a mix of low level and high level features in the language, though it has some things that are quite confusing for a C/Python/Vala guy like myself. Eventually I will post some notes about the synchronization primitives I've used in the implementation of the service that gets the data from Redis every second.

Evolution needs your help!

Evolution just branched for 3.11 starting what would be the latest release under the 6 month cadence. As announced before, Evo is going for a 1 year release cycle that the team thinks is going to enable them to improve things dramatically moving forward.

This also means that there's some pressure to make Evolution 3.12 rock in terms of stability and basic features as this is a version that is going to be used for a whole year by users.

This is why we think this is the right time to join the team and help. If you have specific areas that you want to help with let us know.

We also have a couple of issues we think are an easy one to start if you are already familiar with Gtk+/GObject development in C.

Calendar widget deprecation

We have a Canvas based widget that we would like to replace with either GtkCalendar or a fork of the GtkCalendar widget so that we get all the a11y niceties from a native Gtk+ widget. If you have experience developing Widgets in Gtk+ this should be an easy one, it is also a great opportunity to learn how to do widget development so if you haven't done so in the past, we can definitively help you if you want to take the chance to learn.

Updating icons

A sizable amount of icons are utterly outdated, a gentle artist touch would be greatly appreciated. Notice the Notes and Memos icons, the Attachment and the Appointment icons in the mail list, and also the still-not-symbolic looking glass and brush in the search entry:

GNOME Goals

There are a few GNOME Goals where Evo and/or EDS are lagging behind:

• PortToGMenu
• PortToGtkApplication
• PrintToFile
• Python3Porting
• CorrectIconNames
• ModernAutotools
• UpdateInfoFiles
• DistCheck
• UseTimeoutAddSeconds

Misc.

There are plenty of areas where you can help as well, if you are a UX/UI designer there are plenty of areas in Evo that could use some help like the appointment/meeting editor, here's a list of low hanging fruits:

• Column headers in the mail list with icons should have tooltips
• Clicking on a new arrived email notification does not open that email
• "Mark as important" column should use a greyed out icon

If you are interested in any of these suggestions or if you have other ideas or skills you would like to offer to the community and help making Evo 3.12 rock just join us at #evolution in GIMPNet's IRC or join the evolution-hackers mailing list.

GNOME's official GitHub mirror

GNOME has just launched its official GitHub mirror!

This is an idea that has been in the back of my head for quite a while, my motivation started after I found out that someone fixed my webp pixbuf loader out of the blue without me even advertising the project. That made me realize the value of tracking who forks what and what they do with those forks. Coding and not being in GitHub these days is pretty much like being a photographer and not being in Flickr.

The aim of the mirror is just to allow people to work and publicize their branches and for maintainers to be able to lurk around and find out what are people doing either with branches inside the same repo (through the branch comparison tool) or in other forks.

Issues and wikis are disabled, pull requests can't be disabled but they will be ignored, this mirror has no intention whatsoever to modify the current contribution/review workflow that GNOME has in place.

Other nice side effects is that if our severs and our backup go to /dev/null, we still have a centralized copy of all the repositories.

As per people concerned about the fact that github is a closed source service, I have a few things to highlight: this is just a mirror, we won't rely on them for any of our activities. Also, GitHub has contributed a great deal of their infrastructure code to the free software commons and they offer a free and very valuable service to anyone who has a free software project. I hope that this alleviates any concerns with regards to this topic.

I would like to thank the GitHub staff for being so helpful, Juanje for his help on the initial scripts and Andrea Veri for his support from the GNOME infrastructure side of things!