Click on the image or this link to download the slides.

It was the best we could do.

The most popular question I’ve gotten, when people hear that I went, was “how is it?” Here’s what I say: It’s bad. It’s very bad. The city is ruined. Many streets look normal until you realize that you’re looking at 2nd floors that are resting on the ground. The people are still hungry. One mark of this disaster will be a population with a sizable proportion of amputees. I expect many buildings to be rebuilt with an eye to handicapped access. Another mark will be years of psychological issues. We took to calling it “Earthquake Survivor Syndrome.” A constellation of symptoms – inability to sleep, sour stomach, headaches, body pain, night terrors, and so on. All perfectly reasonable reactions to having your home destroyed, relatives killed, and sleeping on the ground for weeks with no end in sight.

The challenge for Haiti in the next decade will be to leverage the disaster, horrible as it was, into an opportunity. Haiti remains the poorest nation in our hemisphere. It’s perhaps an hour by air from Miami. As I’ve been saying to people over the past four years – this situation is in our backyard.

At some point in the not too distant future, the flavor of the aid that we provide will need to change. Usually, my group charges a nominal fee for clinic visits. Not much – and certainly nowhere near enough to cover our expenses – but charging prevents us from undercutting local care providers. If we always gave away free care, people would simply wait for the free doctors rather than keeping local doctors employed. We want to wind up as a supplement to local capability – not a replacement. Similarly, when we build a school or a clinic – we are very careful to build a Haitian team who are responsible for it – and then provide the resources they need in order to take the lead on construction and operation.

Obviously, on this trip, everything was free. But sometime soon, outright aid needs to become partnership instead. Managing that transition will be a challenge for everyone. This is the difference between the Partners in Health approach, and (for example) the Doctors Without Borders. The latter roll in with team after team of highly trained doctors and staff – but then they roll out again. PiH stays to build capability from the bottom up.

What can we do that would help the most? I honestly do not know.

I do want to give shout-outs to some groups I saw working down there:

* We saw some really high grade shelters and tents provided by Rotary International, Canada, and Coleman. I’m sure that a lot of other groups provided shelters, but those three were everywhere.

* The Dominican Red Cross were everywhere.

* “Save the Children” had truck after truck of supplies rumbling around town.

* The US armed forces were omnipresent – and we heard a rumor that these men and women were volunteering their non-deployment time (i.e: time that they could have been working at a US base and living with their families) to serve in Haiti.

* And of course, Partners in Health.

As of early 2010, Grid Engine is effectively not installable out of the box on modern version of Mac OS X. We have seen this recently with server and client versions of OS X 10.5.* as well as the new Snow Leopard (10.6.*) releases.

This used to not be a big deal and the workarounds were trivial. Something seems to have changed, however, in recent updates to OS X that render the Grid Engine binaries unable to function when :

• When started during the installation process
• SystemStarter() scripts that SGE tries to install on Apple systems
• Manually started by user root via the command line

This is obviously a show-stopper now for new SGE users. We have no idea why this is the case but have witnessed the behavior on many 10.5 and 10.6 systems (many of them running OS X Server).

The problem with Grid Engine on modern versions of OS X can be simply stated:

SGE binaries will not launch or will be unreliable when started by ANY METHOD other than the system level OS X “launchd” service framework.

… and since neither the SGE installation scripts nor “traditional” SystemStarter() scripts that SGE install on Mac OS X systems use launchd framework this basically means that SGE 6.2 is unusable out of the box without manual intervention and custom starter scripts.

How to manually get SGE 6.2x working on Mac OS X systems

This process will be very simple to someone who already understands Grid Engine administration and is comfortable with SGE admin commands – unfortunately it may be confusing to novice or beginners because we have to interrupt the automatic SGE installation process and complete things by hand using SGE admin commands.

The method:

1. Run the “install_qmaster” script normally. The fact that it will fail is not a big deal – before it fails it will construct the SGE_CELL directory, configure spooling and otherwise do all of the behind the scenes steps necessary to support a functional SGE qmaster process.
2. When the install_qmaster script fails, exit out and manually kill any zombie sge_qmaster daemons that may be hanging around on the system
3. Download and run the sge-launchd-scriptmaker tarball from this BioTeam Blog post.
4. Run the sge-launchd-scriptmaker utility. This simple perl script will query your SGE environment and construct several .plist files suitable for copying into the /Library/LaunchdDaemons/ OS X folder
5. Using OS X “launchctl” commands, restart sge_qmaster
6. At this point the SGE qmaster is running and functional and we can manually complete a few additional configuration steps…
7. Create and populate a hostgroup named “@allhosts”
8. Create and configure the default cluster queue object (“all.q”)
9. At this point the SGE install_execd script should work but you can also skip that step and just launchd sge_execd via the launchctl framework (a step that will be necessary even if the exexhost installer script functions fine)

The screencast below shows a recording of me stepping through this process on a small Apple Mac Mini running Mac OS X 10.5.8 Server (sorry, I thought it was a Snow Leopard box when I began the work …).

If you don’t want to watch the embedded video below, you can navigate directly to the screencast site and download the full movie file. The video is hosted here: http://www.screencast.com/t/NjMyNGJiNWM

Feedback welcome.

We drove the FHM vans all the way through to the Dominican Republic, stopped only once by an armed guard. He banged on the window and demanded to know what we thought we were doing just driving 17 people across an international border. “Medicos,” we replied. “Andele con Dios,” he said as he waved us through.

Of course, it wasn’t really that simple. We guessed that we needed to get our passports stamped to get past customs at the airport tomorrow. Hardy and I collected all 15 passports and took them to the Dominican authorities. There was a $10 per passport fee to get the entry visa into the Dominican. They took that (cash, no receipt) and demanded some sort of “medical identification.” He showed them his military ID, which seemed to suffice.

They then revealed that we would need to get an exit visa from Haiti before they could stamp the passport. So we walked back across the border (ignoring a couple of offers to hire a moped for the 200 yard trip), around the exhausted looking guards who waved us through, and to the Haitian authorities. We set a stack of 15 passports on the counter, explained ourselves, and watched them get stamped through.

Back to the Dominican side (hello again to the exhausted guards), received 15 more passport stamps, and returned to the group.

At no point did anyone seem to think that it would be important to see these 15 individuals who were notionally crossing the border. It’s also worth noting that the border crossing is a geographically stunning place, a high mountain lake with chalk cliffs plunging directly into greenish blue water. It was a surreal experience.

This was substantially different than the trip into Haiti, since we bothered to stop at immigration at all. We have neither exit nor entry visas from that trip.

Summary: We’re safely back at the Marriott in Santo Domingo – substantially lighter – and planning to get on a plane home tomorrow.

We’re making do with supplies. Our group leader found a pharmacy and returned with half gallon jugs of liquid Tylenol, Expectorant, and other drugs. This brought our lack of plastic cups to the fore. We’ve been dispensing liquids in doubled ziplock bags. I’ve been enjoying watching the doctors exercise creativity in dealing with the problems we’re seeing using whatever is at hand. For our part, out in the pharmacy, we have a solid system. We have five of us out there. When a patient emerges from treatment, their chart goes on the bottom of a stack. We have two people filling perscriptions, a nurse and a translator handing them out and explaining the drugs, and two of us in the back feeding the pipeline for the other three. We worked flat out with that arrangement from 7:30 this morning through to 4:30 this evening, with a 45 minute lunch break.

We gained a volunteer for the day today. A nurse / EMT from Michigan, down with a group doing construction in Cite Soleil. There’s a very definite camaraderie when we see other Americans – even though we are a bit isolated in our little compound.

In between filling bags with drugs, we watched a man start to demolish his house with a 5lb sledgehammer. He was clever, careful, and absolutely relentless. He worked about the same hours that we did, and got through perhaps a quarter of the top floor.

Tomorrow will be the last day of clinic, and on Saturday we make the drive back through the DR to catch a plane home.

We did clinic this morning, seeing a mere ~150 people. This brings our total for the week to the high 800’s. Honestly, we could be plus or minus fifty. We try to track patients with a rudimentary chart (piece of paper with name, vitals, chief complaint, evaluation by the doctor, treatment administered, drugs prescribed, and so on) but urgent cases pre-empt that process. I’ve also lost count of how many people have been carried past me in a fireman’s (two man) carry as I worked filling small containers with our limited drug supply.

Google maps and Google Earth have remarkable overhead imagery of the city, updated after the quake. I would encourage everyone to take a look at the national palace, the cathedral, and Cite Soleil for reference on what I’m writing here.

Cite Soleil is built on top of the former garbage dump of the city. Not the most stable of foundations – but it’s also explicitly intended for some of the poorest people in the Hemisphere. The poverty there was intense prior to the quake. When it rains, the sewage from the city flows through Cite Soleil on its way to the sea. We stopped at a church supported by the group I’m with (Family Health Ministries). It would normally seat about 2,000 people, but it is unusable due to many cracks through the foundation and walls.

The architect traveling with our team picked up a fist sized piece of concrete from the foundation and crushed it to powder between his hands. “You couldn’t build a doghouse on this foundation anymore,” were his exact words. Apparently once concrete has set, subjecting it to severe compression and strain will weaken it structurally.

We then drove to the national cathedral and the palace. Frankly, I’ve never felt so safe in Port Au Prince before. You can find footage of those buildings on CNN and so on. It’s a lot more intense up close. A city builds a cathedral as an expression of their hopes and dreams – the architectural pinnacle of what they can accomplish. Seeing that reduced to rubble took a lot out of me.

A lot of buildings looked fairly normal, until you realized that you were looking at a second floor resting at street level.

On the other hand, we passed US, Brazilian, and French military and construction convoys. We chatted with several members of other groups who were getting on with the business of building infrastructure.

People are living in clusters of strangely homogenous tents. Here, a hundred from Coleman. There, 200 from the Rotary.

With that, I will again close and let someone else notify family and friends that they are doing okay.

We’re in the ‘Blanchard’ or ‘Terre Noir’ neighborhood, perhaps a mile north of Cite Soleil – in Port au Prince. The clinic building where we are working survived the earthquake in remarkably good shape. It’s in a walled compound that also contains a 1,000+ seat church and a small school. The school is basically a loss, and the church has a large crack from floor to ceiling on the two side walls. The tower that holds both the satellite dish and the water tank is badly damaged, but standing. We suspect that re-filling the water tank would push it over the edge.

Our clinic is providing mostly ambulatory care. Everyone here is in tremendous emotional stress. The majority of the city are sleeping on the streets, and anyone who we can get talking has lost friends and family just a couple of weeks ago. For perhaps 75% of the patients, we’re really providing a friendly face, some painkillers, mild sedatives (benadryl) to help with the nightmares, and so on. A chance to get out of the sun for a few hours and have a couple of cups of clean water is probably a bit of help in and of itself.

That said, we’ve unequivocally saved several lives. We had a man collapse and begin to seize with what was most likely meningitis on Sunday. The doctors started IVs, maintained his airway, and managed to stabilize him. The hospitals refused him, so we wound up having him driven to his house. He came back the next day, walking and talking – and thanking us.

We’ve been working our orthopedic doctor to the bone – in some cases re-breaking and setting limbs that had begun to heal incorrectly. For these folks, we have conscious sedation, which is a real benefit. Still, the screams are quite something.

For my part, I spent a couple of hours yesterday giving two liters of water with rehydration salts to a severely dehydrated girl. One tiny sip per minute, so she wouldn’t vomit it back up. I spend most of my time working in the pharmacy – dispensing medicines and running the few lab tests that we can do (urinalysis, glucose, hemoglobin, and pregnancy).

We’ve seen perhaps 720 people in three days, starting out slow and ramping up to about 270 per day, today and yesterday.

Perhaps a future post can describe the incessant helicopters, and how we’re clearly on the edge of a massively devastated city. For now, food has arrived.

I write this from JFK airport in New York, having successfully gotten out of the house at 5am carrying four duffel bags containing just under 200lbs of (mostly) donated supplies. Our clothing and personal gear are in our carry on luggage.

I remain overwhelmed by the deep vein of compassion and generosity we have tapped. We decided to make this trip less than a week ago, on Sunday the 24th. In the ensuing five days we’ve amassed enough gear and supplies that we spent last night in a sort of triage party.

The 200lb number is far from arbitrary. Not for love or money can you check more than two bags on an international flight. Further, if you’re flying to the Dominican Republic – not for love or money can those bags exceed 50lbs. At the airline’s discretion they may allow overweight bags, for a substantial fee. Since I did not want to see the fruits of people’s generosity thrown out at the gate to make weight, we decided to pre-weigh all of our bags and make a separate shipment of what would not fit.

In terms of shipping, there is still no really coherent way to ship to Port Au Prince. Fedex and UPS will each take packages to the Dominican Republic – but the cost to overnight a 50lb bag was nearly $500. At that price point, I’m sure that there are more useful things to do with the money than spend it on expedited burning of jet fuel. With an additional week’s notice, we could have dropped the cost substantially. So, the remaining supplies are being shipped out this week (by still further friends, with keys to our house), not only out of weight, we were also out of time.

Based on what we see on the news, as well as reports from colleagues and friends in country, we are focusing on pain management, wound care, antibiotics, and postoperative care for the lucky souls who have already received treatment. I have no idea what to do about what is being described as an emerging mental health catastrophe. The hospitals who employ the physicians on the team have been singularly generous. Just the two of us are carrying nearly 50lbs of drugs. We’re also shipping nearly 7,000 over the counter pain pills. The other physicians on the team are similarly laden.

My wife’s hospital came through with a wide assortment of surgical equipment. Apparently in the United States this is considered single-use. Stainless steel implements are simply thrown away. In the rest of the developed world they are sterilized and re-used. We have one bag that is small but dense with retractors, forceps, scalpels, and other things whose names I do not know.

On Wed, we were also allowed into a basement room filled with pallets of donated supplies. “Take what you need, and God bless you,” them man said. We selected an “ambu” breathing bag, as well as lots and lots of gloves and gauze, among other things.

My colleagues at Bioteam, as well as other friends, delivered a total of three boxes of emergency gear (emergency rations, a folding solar power source, tarps, stoves, water purification supplies, and so on) as well as four boxes of home health care supplies. The latter included critical supplies for wound care, splinting, and pain management.

We decided early on that all our clothing would fit in the carry on bags, as well as my communications gear. So, my clothing for the week (including church duds for Sunday) consists of half a duffel bag’s worth of space – crushed next to a satellite internet terminal and a 12″ G4 Apple laptop that will probably live out its days as a clinic computer in Haiti, and a GPS locator beacon.

Food was the hardest compromise. We’re carrying everything we need for a week. That took up critical space – but we decided that it would be unacceptable to make the trip only to become a burden and take supplies that the Haitians need. Perhaps half of one of our larger bags is heat-and-serve pouches of pre-cooked food, trail mix, and so on.

I brought each bag up to nearly 50lbs with the emergency ration bricks – intending to make something of a show of having to leave behind food for the hungry. Fortunately, it was not necessary. In fact, the JetBlue employee who checked us in surprised us by waiving our baggage fees entirely. “You’re doing a good thing,” she said.

Thanks again to all of you who opened your hearts, closets, and supply room doors to us. This is truly humbling. Between the 15 members of our team, we’re bringing 30 bags – 1,500lbs of supplies.

With that, I board a plane bound for the Dominican Republic. Our drivers meet us at the hotel at 7am tomorrow to begin the trip into Port Au Prince.

We have pictures of the guest house where I’ve stayed in the past. If you click the link, you’ll see why we’re sleeping at the clinic this time.

Update:  All of Chris Dwan’s updates and trip reports can be found at http://blog.bioteam.net/tag/haiti/

I’ll be going with a group called Family Health Ministries. We’ll go overland through the Dominican Republic to a clinic where my wife and I have worked previously.

2008 Mission – Clinic Intake Area

It’s in the community of Blanchard (also known as Terre Noir), a mile or two northwest of the core of Port au Prince. This is a community where people who have managed to move out of the absolute slums of Cite Soleil have settled. It’s by no means an affluent neighborhood, but it has been stable enough over the last few years that the group has been able to support a decent clinic and a school there.

I have some pictures of the clinic from my last trip in 2008. They are here.

It’s humbling to note that every place that I ever slept, in three separate visits to Haiti, is now rubble. Of the four clinics, two schools, and one orphanage that our group has helped to build – two clinics and the main school (over 500 students) are beyond repair. We’ll either be sleeping on the floors of the clinic, or else in the open – within a church compound – like the rest of the population.

We hope to provide direct ambulatory care in a small community where we have existing relationships. Our watchword is “walking wounded.” For the larger medical cases requiring surgery, dialysis, or other intensive technologies – we hope to rely on Partners in Health and Doctors Without Borders, both of which are superb organizations.

Our group of 15 includes five physicians and two nurses. Based on previous experience, I will be mostly useless on the medical side of things – except perhaps in terms of organization and communication – as well as perhaps literally carrying water for the doctors. To help me provide some small value, Bioteam is sponsoring a BGAN satellite internet connection to help with both emergency and informational communications. While they’re substantially pricier than a cell phone, I think that the whole team will take comfort from being able to update their friends and families.

Bioteam is a very flexible company. We try not to get bound up in procedures and protocols. Instead we steer by a policy of mutual benefit. If there is a path that is right for our customers, for the company, and for the consultants – that’s almost always the right thing to do. When I broached this opportunity with the rest of the team, they first said “yes, go.” It was only later that they asked if I would be willing to share my experiences on the Bioteam blog.

While I can’t promise anything as technically thrilling as Sun Grid Engine or SSH tunnels – I’ll do my best to convey what it looks like from down there.

Update:  All of Chris Dwan’s updates and trip reports can be found at http://blog.bioteam.net/tag/haiti/