Andrew's PostgreSQL blog - PGExperts

Not a VPN substitute

nospam@example.com (Andrew Dunstan) — Fri, 12 Aug 2011 22:03:00 +0300

Some businesses don't like to go to the bother and expense of installing and maintaining a VPN so that external consultants (or their own workers responding to emergencies in the middle of the night) can connect to machines inside their network. Instead, they like to use network meeting software to share the desktop of a machine inside their network which then talks to the server in question. Vendors of network meeting software actually promote this use as being superior to using a VPN. It isn't. It's slow as heck, horribly cumbersome, and ugly. You are dependent on the machine in question having the software you need. (They don't have pgadmin3 installed? Tough luck.) The price in lost productivity is massive.

Luckily, my most frequent clients don't go in for this nonsense. They use VPN software like OpenVPN, or products from vendors like Juniper or Cisco. When you combine that with things like sshfs, so you can use your favourite editor with your favourite settings, regardless of what rubbish the client might have available (I'm looking at you, pico), this can become a big win.

Working remotely can be hard. If you need external consultants you should plan for it, and expect to provide them with facilities that will enable them to give you the best service they can. I recently spent a week at a client site, and it wasn't until the last day that they approved network access for me, so I was totally disconnected, and reliant on the coffee shop several floors down for even Internet access. It's hard to do your best in such circumstances, and although I did manage to get some big wins for them, I think I could have done much better if getting to their production machines had not been so difficult.

Things fall together

nospam@example.com (Andrew Dunstan) — Sun, 07 Aug 2011 20:17:32 +0300

(apologies to W.B.Yeats)

The other day Hubert Lubaczewski (a.k.a. depesz) blogged about pg_reorg. My PostgreSQL Experts Inc. colleagues have been using it too, and they also like it. This is a utility to perform a cluster operation, i.e reorganize the table, but without requiring an exclusive lock on the table all the time it's running. That means many users who can not use CLUSTER at all today will be able to get the benefit, and many who only can use it at times of scheduled outages will be able to perform this operation much more frequently. That's a huge win.

I've been playing with it and it seems to be everything it's promised to be.

I dropped a note to Itagaki Takahiro a day or so ago asking him about making an updated release, and he quickly responded by releasing version 1.1.7, which contains a fix for the "dropped columns" problem. Many thanks to him for that.

There are one or two things that could be improved. For example, I'd like to see it able to fall back to the Primary Key if there is no clustered index, in effect performing "ALTER TABLE foo CLUSTER ON pkindex" for every affected table that has a primary key but no clustered index. Since it needs a short exclusive lock on the table in order to install the trigger at the start of operations on the table, this should impose no additional lock burden. In the mean time I've cooked up a quick recipe published on github to do the job.

Also, I'm a bit concerned about the fact that when it needs to swap the new table into place it will start cancelling queries and backends with conflicting locks. I'd like to see options for it to back off and retry, or fail, if it can't get the lock at the end. I haven't looked into it, so that might not be possible. But if it's not, users will still need to be very careful about when they run it if they might encounter long-running conflicting operations.

All in all, though, I can tell I'm going to find this very useful. Kudos to the authors for a clever piece of work.

Who pays the piper?

nospam@example.com (Andrew Dunstan) — Tue, 08 Feb 2011 18:25:00 +0200

I was wondering idly today about who pays for community work on PostgreSQL. In particular, who pays for work done reviewing and committing other people's work. I know Bruce, Heikki and Robert are employed by EnterpriseDB, Tom by RedHat and Alvarro by CommandPrompt, and I assume at least some of the time they spend working on this is funded by their employers. I don't know about others, I guess they can speak for themselves. I don't receive any direct funding for Postgres community work. I have in the past received sponsorship for specific features, such as parallel pg_restore and extensible enums, but that doesn't cover time spent on day to day work, or even quite large items such as last year's plperl security fixes, which consumed a huge amount of time.

It could be argued that this is a cost of doing business. I earn my daily bread by working with Postgres, and most or all of my customers come through that connection. That includes PostgreSQL Experts Inc clients. But I could probably drop all or almost all my unpaid community work and not suffer a serious decline in business. I'm not going to, but I wouldn't suffer a lot professionally if I did.

I don't have any grand conclusions out of this. But thoughts like this do come to me every commitfest, when I feel I should be doing more. I will probably pick up three or four more items from the commitfest queue during the next week. But how much I do will be governed by how much time I can snatch from other, paying, activities.

PostgreSQL Experts Inc.

nospam@example.com (Andrew Dunstan) — Thu, 07 May 2009 01:42:47 +0300

When I left the hedge fund I was working for a couple of years ago (boy, does that look like a good decision now), I decided not to seek full time employment any more but rather to try my hand at consulting, concentrating on PostgreSQL work. I formed a company Dunslane Consulting LLC and made my availability known to a few people. Since then I have managed to keep pretty steadily in work, without much effort. I've not had to go looking for work - it has tended to find me at just the right times.

However, the things you can do as a single consultant are limited. Some engagements require a number of roles, and for that I am often either technically or temperamentally unsuited (I have sworn never to work on a GUI again, nor on release management.) That's where PostgreSQL Experts Inc comes in. A number of people have got together to form this company, which is aimed at providing full service to PostgreSQL users, from setup to tuning to administration to application development. We have some people who are pretty well known in the PostgreSQL community, including Josh Berkus, our CEO, and David Wheeler and David Fetter, as well as your humble blogger, so we believe that potential customers will have confidence that they are getting the best when they come to us. Hence our name

Feel free to call me or Josh or any of our other consultants if you need more information. Or check us out at pgCon