I recently started a personal project where I wanted to use a database in the cloud. There are quite a few to choose from. My main criteria was that it be something low or no cost.

Eventually I decided on Firestore, using the Spark Plan. This plan gives me 5Gb of storage, with 50K reads and 20K writes per day for free, which at the time seemed like plenty. I soon learned that a little carelessness can blow past those transaction limits pretty fast.

Firestore is a NoSQL document store database. Each NoSQL database is different and my a learning curve was steeper than expected. As you know, the best teacher is adversity, and I made my share of mistakes early on. One too many, though, and I’d hit the read or write limit of the plan, which sometimes could happen within an hour or two. Then it was time to call it a day.

Things are better now, so I offer these lessons learned:

Do start with the free plan

_Photo by [Unsplash](https://unsplash.com/@frankiefoto?utm_source=ghost&utm_medium=referral&utm_campaign=api-credit">frank mckenna / <a href="https://unsplash.com/?utm_source=ghost&utm_medium=referral&utmcampaign=api-credit)

Yes, it is easy to overshoot the plan’s limits, but those occurrences will force you to learn to be efficient with your reads and writes. You will become more cost-conscious of sequencing multiple database operations in an efficient way.

Start with a small dataset

_Photo by [Unsplash](https://unsplash.com/@rayhennessy?utm_source=ghost&utm_medium=referral&utm_campaign=api-credit">Ray Hennessy / <a href="https://unsplash.com/?utm_source=ghost&utm_medium=referral&utmcampaign=api-credit)

This may seem obvious, but by small I mean less than 100 documents total. On my project, I first created a collection with 10K documents. I later realized that I made a mistake in the data I loaded, fixed that, found another, went to fix that, but… TRANSACTION_RESOURCE_LIMIT_EXCEEDED. Welp, done for the day.

Take time in the design of your data model

_Photo by [Unsplash](https://unsplash.com/@kellysikkema?utm_source=ghost&utm_medium=referral&utm_campaign=api-credit">Kelly Sikkema / <a href="https://unsplash.com/?utm_source=ghost&utm_medium=referral&utmcampaign=api-credit)

Like the carpenter’s axiom: “Measure twice, cut once”, you don’t want to be adjusting your JSON document fields and structure piecemeal. Oh, you will of course, but you’ll save yourself some transactions by practicing a little foresight. Write out a schematic of documents, their fields, and their relationships first. Visualization is the key to happiness.

Test and verify your data loading scripts

_Photo by [Unsplash](https://unsplash.com/@nasa?utm_source=ghost&utm_medium=referral&utm_campaign=api-credit">NASA / <a href="https://unsplash.com/?utm_source=ghost&utm_medium=referral&utmcampaign=api-credit)

You’ll need scripts to populate the database from some other source. That is the time to:

• verify what you loaded is what you expected
• correctly handle the no-data case for fields

I made mistakes in both cases. First, when I loaded some string data into a document field, I hadn’t immediately noticed that those strings had quotes already, so the stored strings had embedded quotes. It didn’t seem that serious an issue, but it became a pain later when writing and testing searches on that field. Because there were a lot of documents, I spent a sizable portion of my daily write quota to clean that up.

In the second case, I discovered that Firestore has no mechanism for determining the existence of a property in a document (there is no undefined check). There is an exists test for documents, but not for document fields. The best practice is to populate missing data fields with null, and then do null equivalence tests in a where clause to find documents with “missing” properties.

What a small dataset won’t teach you

_Photo by [Unsplash](https://unsplash.com/@goodfreephoto_com?utm_source=ghost&utm_medium=referral&utm_campaign=api-credit">Good Free Photos / <a href="https://unsplash.com/?utm_source=ghost&utm_medium=referral&utmcampaign=api-credit)

Once you worked out kinks on the small dataset, it is time to graduate to a larger one. With more documents to process, things like query efficiency, pagination and batch requests become important.

Read in chunks, write in chunks

_Photo by [Unsplash](https://unsplash.com/@picoftasty?utm_source=ghost&utm_medium=referral&utm_campaign=api-credit">Mae Mu / <a href="https://unsplash.com/?utm_source=ghost&utm_medium=referral&utmcampaign=api-credit)

Batch operations allow for multiple read/writes on the database in a single transaction. This means that if any write operation fails, then all writes fail, and the database data retains its original state. Each operation in a batch counts toward the total read/write quotas, so as such it doesn’t help usage quotas. Also, when writing via batch operations, be aware there’s a 500 operation limit per batch.

Be careful when correlating two documents (i.e., for every A document, there is an association by reference to a B document). Don’t fetch all of one first, then iterate through the other. That’s a good way to chew up the transaction quota when debugging.

It is better to fetch a subset of the first collection, then iterate through it document by document. Associate these documents with document in the second collection by fetching one that matches criteria. Continue to do this until the entire first collection has been fetched. When debugging, you can verify everything looks like it is working correctly and, if not, kill the process before a lot of transactions are run.

How to limit query results

_Photo by [Unsplash](https://unsplash.com/@will0629?utm_source=ghost&utm_medium=referral&utm_campaign=api-credit">Will Porada / <a href="https://unsplash.com/?utm_source=ghost&utm_medium=referral&utmcampaign=api-credit)

Firestore’s query language isn’t as richly expressive as SQL is, but there are still a number of ways to restrict your queries so that you don’t overfetch data. Although technically there is no size limit for a POST response body, in practical terms there is.

Some mechanisms for limiting query results:

where and compound where

You can chain multiple where clauses together, similar to adding conditional expressions to a single where clause in SQL.

citiesRef.where('state', '==', 'CO').where('name', '==', 'Denver');

limit and ranges

You can limit the number of documents returned by a query by using chaining a limit clause at the end of the query object.

let biggest = citiesRef.where('population', '>', 2500000)  .orderBy('population').limit(2);

You can also specify a range of records to retrieve via startAt/endAt or startBefore/endBefore constraints, which allows you to do cursor-based pagination.

let docRef = db.collection('cities').doc('SF');

return docRef.get().then(snapshot => {  let startAtSnapshot = db.collection('cities')    .orderBy('population')    .startAt(snapshot);  return startAtSnapshot.limit(10).get();});

in array query

You can look for specific matches in an array. This is good for enumerated values.

const usaOrJapan = citiesRef.where('country', 'in', ['USA', 'Japan']);

As demonstrated, it is possible to work within the limitations of the Spark Plan as you learn about Firestore. It’s free, which is always a good place to start.

I'm interviewing my colleague Lennart Frantzell, a fellow developer advocate on the IBM Developer San Francisco team. Inside IBM, Lennart focuses on one of our most technically in-depth communities: Blockchain.

Q: Your developer advocacy efforts focus on projects with large codebases and technical architectures that requires developers to adapt their mental models. Can you tell us about them?

When I came to Silicon Valley in the late 80s I worked on one of the most exciting projects imaginable, Expert Systems, the precursor to today's AI systems and IBM's Watson project. I spend four years in Menlo Park in the childhood of Silicon Valley creating language compilers with built-in AI capabilities. As often with leading-edge technology, we didn't quite make it. Fortunately I then had the opportunity to work on a team in IBM's Silicon Valley Lab in southern San Jose that took prototypes from IBM Research and productized them. From there I moved to Object Oriented systems and have really always worked on advanced software systems. Some of which succeeded but many of which did not. Just like in the start-up world.

Q: How did you get started with programming?

Originally I was majoring in Asian Studies in Sweden and I received my doctorate from the University of Stockholm. After graduation, I realized that there were no jobs in my chosen field, and I began to take an interest in computers. I took a few courses and landed a job in the IT department of the food chain Shoprite on the East Coast of the US. Two years later IBM was advertising for a job for something called the Series/1 minicomputer, which we used at Shoprite. I was lucky to be accepted and spent the next four years writing assembler code that automated telephone switches in Seattle, which I fell in love with. And which is the reason that my oldest son with his family now lives and works in Seattle.

IBM Series/1 Minicomputer

Q: How did you make the transition from programmer to developer advocate?

I have always loved programming but I have also liked working with people, and when I got the opportunity to jump over to one of IBM's Innovation Centers and work with business partners I didn't hesitate.

After a few years this led me to Developer Advocacy which is really great because one meets so many developers and start ups from various geographies and countries with their own perspective on technology.

Q: How do you find that advocacy for deep technical projects differs from developer advocacy in general?

IT technology is suddenly eating the world as Marc Andreessen has pointed out. It is no longer a peripheral technology but changing society all around us.

One compelling example is AI, with image recognition and facial recognition. We have made amazing strides with increased image recognition capabilities, but it turns out there is a flip side to these advances: do we really want AI-systems that can trace our every step. This is something that we as developer advocates can highlight and make people and ourselves aware of.

There is code but code also has a social impact, both good but also from time to time negative.

I enjoy putting together events that focus both on the technical but also on the social aspects of technology. I enjoy looking at the big picture, and it's exciting to work at a company like IBM that gives me free rein to work with developers and startups and not always follow a specific corporate line.

Q: How do you get introductory programmers interested in a deep or very technical ecosystem?

Start with the impact on society. I get uneasy when I encounter people who may not see the forest for the trees, i.e. the impact their code may have on society.

I should also mention the IBM Code Patterns -- an innovative approach to providing developers with working code around specific themes. They allow us to quickly get developers up and running in areas such as AI, Blockchain or Machine Learning, to name a few areas. Something that makes our job a lot simpler than before and that allows us and the people we talk to to quickly see how new technology is behaving in real life.

Q: What technologies that you’ve worked with are you most excited about?

Blockchain, Machine Learning and AI. These technologies are where the future is, and that are already an important part of our society.

For instance, a self-driving car is something you wouldn’t even imagine a few years ago, and now we’re talking not only about the underlying technology and algorithms, but also the impact on society and the freedom it can bring to people around the world.

Q: Let's talk logistics. What types of events tend to work well for the tech you’re advocating?

Face to face events with between 50 and 100 people present. You must remember that attendees come to these events to mingle. They want to meet people, talk, exchange opinions -- they’re not really interested in listening to us talk for an hour or longer. All information is usually out on the web anyway.

They want to learn and code, sure, but it’s primarily a social event and they want to mingle and get contacts. Being able to read the audience and respond to what they want is an important skill as a developer advocate.

Also, we do a lot of hackathons, which are essentially long-running social events. I love working on hackathons more than almost anything else.

Q: Any best practices in developer advocacy that you’ve been exposed to?

I just went to DevRelCon SF 2019 and you could tell from the presentations that developer advocates as a group are really good at talking to an audience -— especially when they are talking about themselves!

Dana Oshiro from Heavybit gave a great talk at DevRelCon SF 2019. It was very factual, and not specific to developer relations but about different aspects of company fundraising, board dynamics and cashflow that are relevant to advocacy but that many advocates may not be exposed to.

Dana Oshiro speaks at DevRelCon SF 2019

Q: What do you want to learn, and who do you want to meet or partner with?

I want to spend time combining Blockchain, AI, ML and IoT, and see where that will lead. It would also be interesting to spend more time with larger startup companies who are on the way to building enterprise applications — currently, I spend a lot of time with smaller startups and individual developers, but working with larger companies will give you a more holistic view of what companies are trying to build with IBM Cloud-technology, including Blockchain and AI. I’m not working in that space currently but I am hoping to get into it very soon.

To get in touch with Lennart, feel free to reach out on LinkedIn: https://www.linkedin.com/in/lennartfrantzell/

During the OpenStack summit a few weeks ago, I had the chance to talk to some people about my experience on running open source projects. It turns out that after hanging out in communities and contributing to many projects for years, I may be able to provide some hindsight and an external eye to many of those who are new to it.

There are plenty of resource explaining how to run an open source projects out there. Today, I would like to take a different angle and emphasize what you should not socially do in your projects. This list comes from various open source projects I’ve encountered these past years. Let’s explore these bad practices and illustrate them with examples.

Don’t perceive new contributors as an annoyance

When software developers and maintainers are busy, there’s one thing they don’t need: more work. To many people, the instinctive reactions to external contribution is: damn, more work. And actually, it is.

So some maintainers tend to avoid that surplus of work. They explicitly state that they don’t want contributions, or they passive-aggressively make contributors feel un-welcome. This can take a lot of different forms, from ignoring them to being unpleasant. This allows the project maintainer to avoid coaching new contributors on how they can productively add to the project.

This is one of the biggest mistakes you can make in open source. If people are sending you more work, you should do whatever it takes to feel them welcome, so that they will continue working with you. Pretty soon they might become the people doing the work you are doing, instead of you. Just think: retirement!

Let’s take a look at my friend Gordon, who started as a Ceilometer contributor in 2013. He was doing great code reviews, but he was actually giving me more work by catching bugs in my patches and sending me patches that I had to review. Instead of being a bully so that he would stop making me rework my code and review his patches, I requested that we trust him even more by adding him as a core reviewer.

If contributors aren’t able to make their first contribution, they definitely won’t make a second one. They won’t make any. And the project may very well have lost its future maintainers.

Don’t limit your contributors to just grunt work

When new contributors arrive and want to contribute to a particular project, they bring with them a wide range of motivations. Some of them are users, but some of them are just people looking to see what it’s like contributing to an open source project. They want the thrill of giving back to an ecosystem that they use.

The usual response from maintainers is to push people into doing grunt work. That means doing jobs that have no interest, little value, and probably have no direct impact on the project.

Some people actually don’t mind doing grunt work, but others do. Some people will love it as soon as you give them something — anything — to do and acknowledgment for their efforts. But others will feel offended that you’ve asked them to do low impact work. Be aware of this, and be sure to high-five the people doing your grunt work. That’s the only way to keep them around.

Don’t forget to praise even small contributions

When the first pull request a new contributor sends over is just a typo fix, some developers will feel annoyed at the prospect of wasting precious time vetting such a small contribution. And nobody cares about bad English in their documentation anyway, right?

My first contributions to home-assistant and Postmodern were me fixing typos in their documentation.

I contributed to Org-mode for a few years. My first patch to orgmode was about fixing a docstring. Then, I sent 56 patches, fixing bugs and adding fancy features. I also wrote a few external modules. To this day, I’m still #16 on the top-committer list of Org-mode, which lists 390 contributors. So I ended up being much more than a small contributor to their project. I’m pretty sure in retrospect their community was glad that they didn’t dismiss my documentation fix as a waste of their time.

Don’t set the bar too high for new comers

When new contributors arrive, their knowledge about the project, its context, and the technologies can vary largely. One of the mistakes project maintainers often make is to ask contributors to perform a bunch of complicated tasks before they can contribute. This can have the effect of scaring away many contributors who are are relatively shy or introverted. They may just disappear, feeling too stupid to help.

Before making any comment, you should not have any assumption about their knowledge. You also should be very delicate when assessing contributor skills, as some people might feel vexed if you underestimate them too much.

Once their skill level has been properly evaluated (a few exchanges should be enough), you need to mentor your contributor to the right degree so they can blossom. It takes time and experience to master this, and you may likely lose some of them in the process, but it’s a path that every maintainer has to take.

Mentoring is a very important aspect of welcoming new contributors to your project, whatever it may be. Mentorship applies nicely outside free software too.

Don’t make people to make sacrifices in order to help you

This is an aspect that varies a lot depending on the project and context, but it’s really important. As a free software project, where most people will contribute on their own good will and sometimes spare time, you must not require them to make big sacrifices. This won’t work.

One of the worst things you can do is require contributors to fly 5,000 kilometers to meet in person to discuss the project. This puts contributors in an unfair position, based on their ability to leave their family for a week, take a plane/boat/car/train, rent an hotel, etc. This is not good, and you should do everything in your power to avoid requiring contributors to do this in order to participate in the project and feel included in its community.

Don’t get me wrong. I’m not saying that social activities should be prohibited. Quite the contrary. Just avoid excluding people when you discuss any project.

The same applies to any other form of discussion that makes it complicated for everyone to participate: IRC meetings (it’s hard for some people to book an hour, especially depending on the timezone they live in), video-conferencing (especially using non-free software), etc.

Everything that requires people to basically interact with the project in a synchronous manner for a period of time will put constraints on them that can make them uncomfortable.

The best medium is still e-mail and asynchronous derivatives (like bug trackers) which allow people to work at their own pace on their own time.

Don’t forget to that you have a Code of Conduct — written or implied

Conduct codes seem to be a trendy topic (and a touchy subject), as more and more communities open up to a wilder audience than before — which is great.

Actually, all communities have a code of conduct, whether it’s written with black ink, or being carried in everyone’s mind unconsciously.

Now, depending on the size of your community and how you feel comfortable applying it, you may want to write it down in a document, like Debian did.

Having a code of conduct does not transform your whole project community magically into a bunch of care bears who follow its guidance. But it does provide an interesting artifact that you can refer to when needed. You can throwing it at some people, to indicate that their behavior is not welcome in the project, and somehow, ease their potential exclusion — even if nobody wants to go that far generally, and that’s it’s rarely that useful.

I don’t think it’s mandatory to have such a paper on smaller projects. But you have to keep in mind that the implicit code of conduct will be derived from your own behavior. The way your leader(s) communicate with others will set the entire social mood of the project. Do not underestimate that.

When we started the Ceilometer project, we implicitly followed the OpenStack Code of Conduct before it even existed, and probably set the bar a little higher. Being nice, welcoming and open-minded, we achieved a decent diversity score, with up to 25% of our core team being women — way above the current ratio in OpenStack and most open source projects!

Don’t make non-native English speakers feel like outsiders

It’s quite important to be aware of that the vast majority of free software projects out there are using English as the common language of communication. This makes a lot of sense: it’s a commonly spoken language, and it seems to do the job well.

But a large proportion of the hackers out there are not native English speakers. Many are not able to speak English fluently. This means that the rate at which they can have a conversation might be slow, which can frustrate native English speakers.

The principal demonstration of this phenomena can be seen in social events (e.g. conferences) where people are debating. It can be very hard for people to explain their thoughts in English and to communicate properly at a decent rate, making the conversation and the transmission of ideas slow.

The worst thing that one can see in this context is a native English speaker cutting people off and ignoring them, just because they are talking too slowly. I do understand that it can be frustrating, but the problem here is not the non-native English speaking — it’s that the medium being used does not put your fellow coders on an the same even playing field that asynchronous conversation would.

To a lesser extent, the same applies to IRC meetings, which are relatively synchronous. Completely asynchronous media do not have this flaw, and that’s why they should also be preferred.

No vision, no delegation

One of the most common tragedies in the open source world is seeing a maintainer struggling with the growth of their project while other contributors are unsuccessfully trying to help them.

Indeed, when an influx of new contributor starts asking for new features, feedback, or directions, some maintainers choke and don’t know how to respond. This ends up frustrating contributors, who therefore may simply vanish.

It’s important to have a vision for your project and to communicate it. Make it clear to your contributors what you want and don’t want in your project. Transferring that in a clear (and non-aggressive, please) manner, is a good way of lowering the friction between contributors. They’ll pretty soon know whether they want to join your ship or not, and what to expect. So be a good captain.

If they chose to work with you and contribute, you should start trusting them as soon as you can, and delegate some of your responsibilities. These can be any tasks that you used to do yourself: review patches targeting some subsystem, fixing bugs, or writing docs.

Let people own an entire part of the project, so they feel responsible and care about it as much as you do. Doing the opposite — being a control-freak — is your best shot at staying alone with your open source software. And no project is going to grow and become successful that way.

In 2009, when Uli Schlachter sent his first patch to awesome, this was more work for me. I had to review this patch, and I was already pretty busy designing the new versions of awesome and doing my day job! Uli’s work was not perfect, and I had to fix it myself. More work. And what did I do? A few minutes later, I replied to him with a clear plan of what he should do, and what I thought about his work.

In response, Uli sent patches and improved the project. Do you know what Uli does today? He manages the awesome window manager project, and has since 2010 instead of me. I managed to transmit my vision, delegate it, and then retire!

Be careful to recognize all kinds of contributions

People contribute in different ways, and it’s not always with code. There are a lot of tasks surrounding free software projects: documentation, bug triage, user support, user experience design, communication, translation…

For example, it took a while for Debian to recognize that they should grant their translators the status of Debian Developer. OpenStack is working in the same direction by trying to recognize non-technical contributions.

As soon as your project starts attributing badges to some people and creating different classes of members in the community, you should be very careful that you don’t forget anyone. That’s the easiest road to losing contributors along the road.

Don’t forget to be thankful

This whole list has been inspired by many years of open source hacking and free software contributions. Let me know in the comments section if you have anything that has blocked you from contributing to open source projects. Thanks for reading!

Original article at https://julien.danjou.info/blog/2016/foss-projects-management-bad-practice