Intro  

  One of the most asked questions related to the novel coronavirus is "what is the mortality rate of the disease?" And most medical professionals and statisticians will choose not to answer it, because so far the data is not consistent enough to tell. Various countries report things differently, have different testing rates and methods and probably different definitions of what it means to be dead or recovered from Covid-19. To give a perfectly informed answer to this is impossible and that is why the people we look to for answers avoid the question, while people who are not professionals are giving all of the possible answers at the same time. I am not a professional, so I can give my answer and you can either trust my way of thinking or not.

  In order to compute mortality with absolute certainty we need several things:

  • the pandemic has to be over
  • the number of deaths from SARS-Cov-2 has to be exactly known
  • the number of people infected with SARS-Cov-2 has to be exactly known

 Then the answer would be the total number of dead over the total number of infected people (100*dead/infected). During the epidemic, though, people tend to use the numbers they have.

Panic!

 One of the most used formulas is: current number of deaths over the total number of infected so far (100*current deaths/current infected). This formula is wrong! Imagine there would be two people A and B. Both get infected at the same time and no one else gets infected after that. A will die from the disease in a week, B will recover in two weeks. If we use the formula above, for the first week the mortality of the disease is 0, then it becomes 50% after a week and it stays that way until the end. If B would die, too, the mortality would be computed as 0, then 50, then 100%. As you see, not much accuracy. In the case of Covid-19 the outcome of an infection is not known for three weeks or even more (see below).

  But let's use it anyway. Today, the 31st of March 2020, this would be 100*37832/786617 which is 4.8%. This is a large number. Applied to the entire world population, it would yield 362 million deaths.

  Accuracy comes from the finality of an outcome. A dead man stays dead, a recovered one stays recovered. A better formula is current number of deaths over the sum of current number of deaths and current number of recovered (100*current deaths/(current deaths+current recovered)). This eliminates the uncertainty of people who are sick, but still have to either die or live. If we would know with certainty who is infected and who is not, who died from Covid-19 and who recovered, this would actually be pretty accurate, wouldn't it?

  If we use it on Covid-19 today, we have  100*37832/(37832+165890), which gives us an 18.57% mortality rate. "What!? Are you insane? That's a fifth of all people!", you will shout, with immediate thoughts of a fifth of the world population: 1.4 billion people.

  So which number is the correct one? Neither. And it all comes from the way we define the numbers we use.

Reality

  OK, I kind of tricked you, I apologize. I can't answer the question of mortality, either. My point is that no one can. We can estimate it, but as you have seen, the numbers will fluctuate wildly. And the numbers above are not the extremes of the interval, not by a long shot. Let's explore that further while I explain why numbers derived from bad data cannot be good data.

  What are the types of data that we have right now?

  • deaths
  • infected (cases)
  • recovered
  • tested
  • total population of an area

  And we can't trust any of these.

Cases/infected

  One cannot confirm an infection without testing, which is something that for most countries (and especially the ones with numerous populations) it is really lacking. We know from small countries like Iceland that when you test a significant part of the population, half of the number of infections show no symptoms. The rest of 50% are on average also experiencing mild symptoms. The number of severe cases that can lead to death is relatively small. The takeaway here is that many more people can be infected than we think, making the actual mortality rate be very very small.

  So, can we use the Iceland data to compute mortality? Yes we can, on the part of the population of Iceland that was tested. We can't use that number for anything else and there are still people that have not been infected there. What is this significant percent of the population that was tested? 3%. 3% right now is considered large. Iceland has a population of 360000, less than the neighbourhood I live in. 3% of that is 108000 people. The largest number of tests have been performed in South Korea, a staggering number of 316664. That's only 0.6% of the total population size.

  But, using formula number 2, mortality for from the Iceland data would be 100*2/(2+157), which is 1.26%. Clearly this will get skewed quite a lot if one more person dies, so we can't really say anything about that number other than: yay! smaller than 4.8%!

  We can try on South Korean data: 100*162/(162+5408) which gives us a 2.9% mortality rate.

  Yet, assuming we would test a lot of people, wouldn't that give us useful data to make an accurate prediction? It would, only at this time, testing is something of a confusing term.

Testing

  What does testing mean? There are two types of tests: based on antibodies and based on RNA, or molecular tests. One tells you that the body is fighting or has fought an infection, the other is saying that you have virus stuff in your system. The first one is faster and cheaper, the other takes more time, but is more accurate. In all of these, some tests are better than others.

  There were reports that people who were infected before and recovered got reinfected later. This is not how that works. The immune system works by recognizing the intruder and creating antibodies to destroy it. Once your body has killed the virus, you still keep the antibodies and the knowledge of what the intruder is. You are, for at least a number of months, immune to the virus. The length of time for this immunity depends not on how forgetful your immune system is, but on how much the virus mutates and can trick it into believing it is not an intruder. As far as we know, SARS-Cov-2 is relatively stable genetically. That's good. So the reason why people were reported to get reinfected was that they were either false positives when they were detected or false negatives when they were considered recovered or false positives when they were considered reinfected.

  Does it mean we can't trust testing at all and it's all useless? No. It means that at the beginning, especially when everybody was panicking, testing was unreliable. We can start trusting tests now, after they have been used and their efficacy determined in large populations. Remember, though, that the pandemic pretty much started in January and for many countries just recently. It takes time to make this reality the new normal and people and technology work in a "proper way".

  Testing is also the official way of determining when someone has recovered.

Recovered

  It is surprisingly difficult to find out what "recovered" means. There are also some rules, implemented in a spotty way by the giants of the Internet, which determine which web pages are "not fake news", but I suspect that the system filters a lot of the legitimate ones as well. A Quora answer to the question says "The operational definition of “recovered” is that after having tested positive for the virus (you have had it) you test negative twice, 3 days apart. If you test negative, that means that no RNA (well, below a certain threshold) from the virus is found in a nasal or throat swab."

  So if you feel perfectly fine, even after having negative effects, you still have to test negative, then test negative again after three days. That means in order to determine one is recovered, two tests have to be used per person, tests that will not be used to determine infection in people with symptoms or in people who have died. I believe this would delay that kind of determination for quite a while.

  In other words, probably the number of recovered is way behind the number of infected and, obviously, of deaths. This means the mortality has to be lower than whatever we can compute using the currently reported values for recovered people.

Deaths

  Surely the number of dead is something we can trust, right? Not at all. When someone dies their cause of death is determined in very different ways depending on where they died and in situations where the morgues are overflowing with dead from the pandemic and where doctors are much better used for the sick you cannot trust the official cause of death! Moreover, on a death certificate you can write multiple causes of death. On average, they are about two or three, some have up to 20. And would you really use tests for covid for the dead rather than for the sick or recovered?

 Logically it's difficult to assign a death to a clear little category. If a person dies of a heart attack and it is tested positive of SARS-Cov-2, is it a heart attack? If someone dies of hunger because they lost their job during the pandemic, is it a Covid-19 death or not? If an 87 year old dies, can you really say which of the dozen medical conditions they were suffering of was the culprit?

 So in some situations the number of deaths associated with Covid-19 will be overwhelmingly exaggerated. This is good. It means the actual mortality rate is lower than what we can determine right now.

Population in an area

  Oh, come on! We know how many people there are in an area. How can you not trust this? Easy! Countries like China and Italy and others have implemented quarantine zones. That means that the total people in Italy or China is irrelevant as there are different densities of the contagion in regions of the same territory. Even without restrictive measures, geography and local culture as well as local genetic predispositions will work towards skewing any of the relevant values.

  Yeah, you can trust the number of people in small areas, especially if they are isolated, like Iceland, but then you can't trust those numbers in statistics, because they are not significant. As the virus spreads and more and more people get infected, we will be able to trust a little more the values, as computed over the entire world, but it will all be about estimations that we can't use in specific situations.

Infectiousness

  An important factor that will affect the total number of deaths, rather than the percent of dead over infected, is how infectious Covid-19 really is. Not all people exposed to SARS-Cov-2 will get infected. They are not really immune, some of them will be, some of them will be resistant enough to not catch the virus. I need a medical expert to tell me how large this factor is. I personally did not find enough information about this type of interaction (or lack thereof) and I suspect it is a small percent. However, most pessimistic scenarios assume 80% of the world population will get infected at some point. That implies a 20% that will not. If anyone knows more about this, please let me know.

Mortality trends

  There is another thing that has to be mentioned. By default viruses go through the process of attenuation when going through large populations. This is the process by which people with milder symptoms have a larger mobility, therefore they infect more people with their milder strain, while sicker people tend to "fall sick" and maybe die, therefore locking the more aggressive strains away from the general population. In this context (and this context only) quarantines and social distancing are actually bad because they limit the mobility of the milder strains as well as of the aggressive ones. In extreme cases, preventing people from interacting, but then taking severely sick people to hospitals and by that having them infect medical personnel and other people is making the disease stronger.

  However, statistically speaking, I expect the mortality of the virus to slowly decrease in time, meaning that even if we could compute the mortality rate exactly right now, it will be different later on.

  What about local authorities and medical administrators? How do they prepare for this if they can't pinpoint the number of sick and dead? The best strategy is hope for the best while preparing for the worst. Most politicians, though, live in a fantasy world of their own making where words and authority over others affect what and how things are done. There is also the psychological bias of wanting to believe something so much that you start believing it is probable. I am looking at you, Trump! Basically that's all he does. That being said, there are a lot of people who are doing their job and the best they can do is to estimate based on current data, then extrapolate based on the evolution of the data.

  So here is another piece of data, or rather information, that we have overlooked: the direction in which current data is moving. One of the most relevant is what is called "the peak of the contagion". This is the moment when, for whatever reasons, the number of infected and recovered has reached a point where the virus has difficulties finding new people to infect. The number of daily infections starts to decrease and, if you can't assign this drop to some medical or administrative strategy, you can hope it means the worst is behind you. Mind you, the number of total infected is still rising, but slower. I believe this is the one you should keep your attention on. While the number of daily infected people increases in your area, you are not out of the woods yet. 

Mechanism

  Statistical studies closely correlate the life expectancy of a population with the death rate in that population. In other words there isn't a specific mechanism that only kills old people, for example. In fact, this disease functions like a death probability amplifier. Your chances to die increase proportionally to how likely you were to die anyway. And again, statistically, it doesn't apply to you as an individual. The virus attacks the lungs and depending on your existing defenses, it is more or less successful. (To be fair, the success of a virus is measured in how much it spreads, not how badly it sickens its host. The perfect virus would show no negative symptom and increase the health or survival chances of its host. That's how vampires work!)

  I have no doubt that there are populations that have specific mutations that make them more or less susceptible to SARS-Cov-2, but I think that's not statistically relevant. I may be wrong, though. We can't know right now. There are reports of Italian regions in the middle of the contagion that have no sick people. 

Conclusion

  We cannot say with certainty what is the mortality rate right now. We can't even estimate it properly without going into horrible extremes. For reasons that I cannot ascertain, WHO Director-General Dr Tedros Adhanom Ghebreyesus announced on the 3rd of March a mortality rate estimated at 3.4%. It is immense and I personally believe it was irresponsible to make such a statement at that time. But what do I know?

  My personal belief, and I have to emphasize that is a belief, no matter how informed, is that the mortality of this disease, by which I mean people who would have not died otherwise but instead died of viral pneumonia or organ failure due to SARS-Cov-2 overwhelming that very organ over the total people that have been exposed to the virus and their immune system has fought it, will be much less than 1%. That is still huge. Assuming a rate of infection of 80%, as many scenarios are considering right now, that's up to 0.8% of all people dying, meaning 60 million people. No matter what proportion of that number will die, it will still be a large number.

  The fact that most of these people would have been on their way anyway is not really a consolation. There will be loved grandparents, people that had various conditions and were happily carrying on with their first world protected lives, believing in the power of modern medicine to keep them alive. I really do expect that the average life expectancy, another statistic that would need thousands of words to unpack, will not decrease by a lot. In a sense, I believe this is the relevant one, though, in terms of how many years of life have been robbed from people by this virus. It, too, won't be easy to attribute. How many people will die prematurely because of losing their job, not getting medical attention when they needed it, getting murdered by people made insane by this whole thing, etc?

  Also, because the people who were more likely to die died sooner, or even got medical attention that they would otherwise not gotten, because pollution dropped, cars killed less people, etc, we might actually see a rise of the life expectancy statistic immediately after the pandemic ends.

  Bottom line: look for the daily number of newly infected people and rejoice when it starts consistently decreasing. After the contagion, try to ascertain the drop in average life expectancy. The true effects of this disease, not only in terms of mortality, will only become visible years after the pandemic ends.

  I didn't want to write about this. Not because of a false sense of security, but because everybody else talked about it. They all have opinions, most of them terribly wrong, but for me to join the fray and tell the world what I think is right would only put me in the same category as them. So no, I abstained. However, there are some things so wrong, so stupidly incorrect, that I can't maintain this silence. So let's begin.

  "The flu", "a cold" are not scientific, they are popular terms and they all relate to respiratory infectious diseases caused by a variety of viruses and sometimes bacteria or a combination thereof. Some of them affect us on a seasonal basis, some of them do not. Rhinoviruses are the ones most often associated with the common cold and they are seasonal. However, a whooping 15% of what is commonly called "a cold" comes from coronaviruses, thus named because of their crown-like shape. Influenza viruses, what we would normally call "flu" are a completely different type of virus. In other words, Covid-19 is more a common cold than a flu, but it's not the seasonal type. Stop wishful thinking that it will all go away with the summer. It will not. Other famous coronavirus diseases are SARS and MERS. The SARS epidemic lasted until July, the MERS epidemic spreaded just fine in the Middle Eastern summer weather. This will last. It will last for months from the moment I am writing this blog. This will be very important for the next section of the post.

  Also, there is something called the R-naught (R0), the rate with which a virus spreads to other people. It predicts, annoyingly accurate, how a disease is going to progress. This virus has an R0 probably twice as high as that of the influenza virus, which we all get, every fucking year. Draw your own conclusions.

  The only reason we got rid of SARS and MERS is because they are only infectious after the symptoms are apparent and the symptoms are pretty damn apparent. Covid-19 is very infectious even before the first cough, when people feel just fine. Surely masks will help, then? Not unless they are airtight. Medical masks are named so because medics use them in order to not cough or spit or breathe inside a patient, maybe during surgery. The air that the doctor breathes comes from the sides of the mask. So if you get sick and you wear the mask it will help the people that have not met you while you had no symptoms yet.

  Washing the hands is always good. It gets rid of all kind of crap. The primary medium of spreading Covid-19 is air, so you can wash your hands as often as you'd like, it helps very little with that. Stopping touching your face does little good, either. There is a scenario when someone coughs in their hand, touches something, then you touch it, then you pick your nose. Possible, so it's not all worthless, it's just statistically insignificant. What I am saying is that washing your hands and not touching yourself decreases the probability a very small amount. That being said, masturbation does increase the activity of your immune system, so be selective when you touch yourself.

  The idea that old people are the only ones affected is a myth. Age statistically correlates with harsher symptoms because it also correlates with negative health conditions. In other words, people with existing health conditions will be most affected. This includes smokers, obese people, people with high blood pressure, asthma and, of course, fucking old people. The best way to prepare for a SARS-Cov-2 virus (the latest "official" name) is to stay in good health. That means healthy food, less alcohol, no smoking and keeping a healthy weight. So yes, I am fucked, but at least I will die happy... oh, no, I am out of gin!!

  Medically, the only good strategy is to develop a vaccine as soon as possible and distribute it everywhere. It will lead quicker and with less casualties to the inevitable end of this pandemic: when more people are immune than those who are not. This will happen naturally after we all get infected and get healthy (or die). All of the news of people who got sick after getting healthy are artefacts of defective testing. All of it! Immunity does not work like that. You either got rid of it and your body knows how to defend itself or you never had it or you had something else or somebody tested you wrong.

  That being said, fuck all anti-vaxxers. You are killing people, you assholes!

  Personally, the best you can do is keep hydrated and eat in a balanced way. You need proteins and zinc and perhaps vitamin C (not sure about that). Warm bone broths will be good. Zinc you get from red meat and plant seeds. There was a report of drinking green tea being negatively correlated with influenza infections (different virus, though). And don't start doing sport now, if you haven't been doing it already, you can't get the pig fat one day before Christmas. Sport is actually decreasing the efficiency of your immune system.

  This is the end of the medical section of this post. There is nothing else. Probiotics won't help, Forsythia won't help, antibiotics will certainly not help. The only thing that fights the virus right now is your immune system, so just help it out. If there was a cure for the common cold you wouldn't get it each year every year.

  But it's not over. Because of people. When people panic, bad things happen. And by panic, I mean letting their emotions get the better of them, I mean not thinking people, not zombie hordes, although sometimes the difference is academic.

  Closing schools and workplaces and public places has one beneficial effect: it makes the infection rate go down. It doesn't stop the spread, it doesn't stop the disease, it just gives more time to the medical system to deal with the afflicted. But at the same time, it closes down manufacturing, supply chains, it affects the livelihood of entire categories of people. So here is where governments should step in, to cover financially the losses these people have to endure. You need money for medical supplies and for keeping healthy. Think of it as sponsoring immune systems.

  The alternative, something we are seeing now in paranoid countries, is closing down essential parts of national economies with no compensation. This is the place and time for an honest cost vs. gain analysis. Make sure the core of your nation is functioning. This is not one of those moments when you play dead for a few minutes and the bear leaves (or falls down next to you because he really likes playing that game). This is something that needs to work for months, if not a year or more. This is not (and never was) a case of stopping a disease, but of managing its effects. Some people are going to die. Some people are going to get sick and survive. Some lucky bastards will cough a few times and go on with their day. Society and the economical system that sustains it must go on, or we will have a lot more problems than a virus.

  Speaking of affected professions, the most affected will be medical personnel. Faced day in and day out with SARS-Cov-2 infections they will get infected in larger numbers than the regular population. Yes, they will be careful, they will wear masks and suits and whatever, but it won't help. Not in a statistical way, the only way we must think right now. It's a numbers game. It's no longer about tragedies, it's about statistics, as Stalin used to say. And these people are not regular people. They've been in school for at least a decade before they can properly work in a hospital where Covid-19 patients will be admitted. You lose one of these, you can't easily replace them. Especially in moron countries like my own, where the medical system is practically begging people to leave work in other countries. The silver lining is that probably, at the end of the outbreak, there will be a lot more medical people available, since they went through the disease and emerged safe and immune. But there is a lot of time between now and then.

  Closing borders is probably the most idiotic thing one can do, with perhaps the exception of countries that had real problems with immigration before. If sick people don't crowd your borders in order to take advantage of your medical system, closing borders is just dumb. The virus is already in, the only thing you are stopping is the flow of supplies to handle the disease. Easter is coming. People from all over the world will just move around chaotically to spend this religious holiday with their family. It will cause a huge spike in the number of sick people and will probably prompt some really stupid actions taken by governments all over the place. One could argue that religion is dumb at all times, but right now it makes no difference. It's just an acceleration of a process that is already inevitable, Easter or no Easter.

  Statistics again: look at the numbers and you will see that countries get an increase of 30% in infected cases every day. It's an exponential curve. It doesn't care about your biases, your myths, your hopes, your judging. It just grows. China will get infection cases as soon as travelling restrictions relax. Consider the ridiculous situation where one somehow protected their country against infection when the whole of the world went through a global pandemic. It doesn't even matter. It's not even healthy, as sooner or later that virus will affect only them. The best they can do is manage the situation, bottleneck it so that the medical system can cope with it.

  Do you know what the most important supply chain is in this situation? Medical supplies. A lot of countries get these from China and India. Because they are cheaper. So they can sell them to you at ten times the prices and make those immense profits that generated the name Big Pharma. It's not a conspiracy theory, it's common knowledge. What do you think happens when you close your borders with China and India?

  In this situation, the globally economy will stagger. It will be worse than the 2008 crisis. But while that was a crisis generated by artificial and abstract concepts that affected the real economy, that of people working for other people, this one comes as real as it gets, where people can't work anymore. That means less money, less resources, scarcity of some resources, less slack to care of the old and sick in your family. It's a lose-lose situation: the most affected by the pandemic will be affected either by people not being able to care for them or people giving them the disease while caring for them because they must make much more effort and human contact to get the supplies needed. Now, some countries can somehow handle that by employing a healthy transport infrastructure and care system, but in others, where they can barely handle normal quantities of sick people that come to hospitals themselves, they will never be able to cover, even if they wanted to, the effort to give supplies to previously affected people.

  So does that mean you have to go to the supermarket and get all the supplies you might need for months to come? I am afraid to say that it does. The reasonable way to handle this is for the governments of the world to ensure supply and financial support for everybody. Then people wouldn't need to assault shops to get the last existing supplies. If you can trust your government to do that, by all means, trust you will always have a nearby shop to sell you the goods you need to stay alive and health. But I ask you this: if you got to the farmacy and bought their entire stock of some medicine that you might need and then you hear your neighbor, the person you greeted every day when you got to work, died because they couldn't get that medicine, what then? What if you hear they need the medicine now? Will you knock at their door and offer it to them? Maybe at five time the price? Or maybe for free? What if you are the neighbor?

  And you hear that some country has isolated the virus and are making a vaccine. Oh, it's all over, you think. But before they even start mass producing it, they need to test it. People will die because of how overcautious and bureaucratic the system is. People will die when corners are cut. People will die either way. It will take time either way. This thing will be over, but not soon. After they make it, you will still have to get it. That means supply chains and money to buy things.

  Bottom line: it's all about keeping systems going. In your body, the immune system has to be working to fight the disease. In your country, the economy must be working in order to handle the effects of the disease. Fake cures and home remedies are just as damaging as false news of the crisis not being grave, getting over soon or going away by itself.

  Here is a video from a medical professional that is saying a lot of the things I've listed here:

[youtube:E3URhJx0NSw]

  One more thing: consider how easy it was for this panic to lead to countries announcing national emergency, a protocol that gives extraordinary powers to the government. A few dead here, a few sick there, and suddenly the state has the right to arrest your movement, to detain you unconditionally, to close borders, to censor communications. Make sure that when this is over, you get every single liberty back. No one it going to return freedom to you out of their own good will.

Summary

Once you finished with the foundation, it doesn't matter who you call to architect your house or fix problems you might have. Businesses and software are exactly like that. Think hard about your foundation, it will save you a lot of effort later on. I've been working in a lot of different places and was surprised to see they didn't know there are other ways of doing things. I distill the foundational principles one needs for a good software solution and maybe not just software:

  • Separation of concerns - processes, components and people should be able to function in isolation. If you can test they work when everything else is shut down, you're good. People should only do what they are good at. Components should do only one thing.
  • Cleanliness - keep your code readable rather than efficient, your process flow intuitive, roles and responsibilities clear. Favor convention over documentation, document anything else.
  • Knowledge sharing - Allow knowledge to be free and alive in your organization by promoting knowledge sharing, collaborative documentation, searchability.

Intro

  I am not the greatest of all developers or architects, but I am good. I know how things should be and what they should do in order to reach a goal. When people ask me about software, my greatest gaps are around specific software tools or some algorithm, not the general craft. That is because of several reasons: I enjoy my work, I've been really enthusiastic in my youth and sponged everything I could in order to become better and I've worked in many different types of organizations so I know multiple ways in which people have tried to do this. As I grow older, the last one may be my most valuable skill, but I am yet to find the employer to realize this.

  You see, what I've learned from my career so far is that most businesses live in a bubble. Used to not only learn software development as I am working on some task, but also network with other people in the craft from all across the business, I kind of expected every other developer to be like that. Or at least the senior devs, the dev leads and architects, maybe some managers. But no, most of the time, people are stuck in their little position and never stray from it. They may invoke life work balance, or they are just burned out, or they just don't care enough, but more often, they haven't even realized what they are missing. And that's the bubble. A bubble is not a prison, is just a small area in which people stay voluntarily and never get out of.

  This is why gaming development is so different from business app development. That is why development in an administrative business with a small software department is so different from development in a software company. It is why development in small shops is so different than in large software companies. Sometimes people, really smart people, have worked for a long time in only one of these ecosystems and they only know that one. They can hardly conceive of different ways to do things.

  So this is why I am writing this post, to talk about the foundations of things, that part that separates one from the other, forever, no matter what you do afterwards. And this applies to business, people organization, but especially well to large software projects. You see, if you start your solution badly, it will be bad until you rewrite it. Just like a building with a weak foundation. It doesn't matter you bring the best workers and architects afterwards, they will only build a wonderful house that will fall down when the foundation fails. You want to make a good thing, first plan it through and build the greatest foundation you can think of and afford. It's much easier to change the roof than the foundation.

  And you wouldn't believe how many times I've been put in the situation of having to fix the unfixable. "Hey, you're smart, right? We started this thing a million years ago, we thought we would save some money, so we got a bunch of junior developers to do it, and it worked! But then it didn't anymore. Fix it!" And I have to explain it to them: you can't scale duct tape. You can go only so much with a thing held together by paper clips, chewing gum and the occasional hero employee with white hair and hunched back and in his twenties.

  Now of course, to an entitled senior engineer like me any software evokes the instinct to rewrite it in their own image. "Also, get some juniors to carve my face into that hill over there!". Sometimes it's just a matter of adapting to the environment, work with what you have. But sometimes you just have to admit things are beyond salvation. Going forward is just a recipe for disaster later on. It's the law of diminishing returns when the original returns were small to begin with. And you wouldn't believe how many people agree with that sentiment, then declare there is nothing that can be done. "They won't give us the budget!" is often muttered. Sometimes it's "We only need this for a few years. After that we start from scratch" and in a few years some "business person" makes a completely uninformed cost and gain analysis and decides building up from existing code is cheaper than starting over. But don't worry, they will suuuurely start from scratch next time.

  Sometimes the task of rewriting something is completely daunting. It's not just the size of it, or the feeling that you've achieved nothing if you have to start over to do the same thing. It's the dread that if you make the same thing and it takes less effort and less money and it works better then you must be inferior. And it's true! You sucked! Own it and do better next time. It's not the same thing, it's version 2.0. You now have something that you couldn't have dreamed of when building version 1.0: an overview. You know what you need, not what you think you will need. Your existing project is basically the D&D campaign you've played so many times that it has become a vast landscape, rich with characters and story. You've mapped it all down.

  This post is an overview. Use it! To be honest, reaching this point is inevitable, there will always be a moment when a version 2.0 makes more sense than continuing with what you have. But you can change how terrible your software is when you get to it. And for this you need the right foundation. And I can teach you to do that. It's not even hard.

Separation of Concerns

  Most important thing: separation of concerns. Components should not be aware of each other. Compare a Lego construction to a brick and mortar one. One you can disassemble and reassemble, adding to it whatever you need, the other you need to tear down and rebuild from zero. Your foundation needs to allow and even enable this. Define clear boundaries that completely separate the flow into independent parts. For example a job description is an interface. It tells the business that if the person occupying a job leaves, another can come and take their place. The place is clearly defined as a boundary that separates a human being from their role in the organization.

  Software components, too, need to be abstracted as interfaces in order to be able to swap them around. And I don't mean the exact concept of interface from some programming languages. I mean that as loosely as one can. A web service is an interface, since it abstracts business logic from user interface. A view model is an interface, as it abstracts the user interface logic from its appearance. A website is an interface, as it performs a different task than another that is completely separated. If you can rewrite an authorization component in isolation and then just replace the one you have and the application continues to work as before, that means you have done well.

  Separation of concerns should also apply to your work process and the people in it. A software developer should not have to do much outside developing software. A manager should just manage. People should only be in meetings that bring value and should only be in those that actually concern them. If the process becomes too cumbersome, split it up into smaller pieces, hire people to handle each of them. Free the time of your employees to do the job they are best suited for. 

  One important value you gain from isolating components is testing. In fact, you can use testing as a force towards separation of concerns. If you can test a part of your application in isolation (so all other parts do not need to be working for it), then you have successfully separated concerns. Consider a fictional flow: you get on the bus, you get to the market, you find a vegetable stand, you buy a kilo of tomatoes, you get back to the bus, you come home. Now, if you can successfully test your ability to get on a bus, any bus, to get anywhere the bus is going, in order to test that you can buy tomatoes from the market you just test you can find the stand and buy the tomatoes. Then, if you can test that you can buy anything at any type of stand, you only need to test your ability to find a stand in a market.

  It seems obvious, right? It feels that way to me. Even now, writing this post, I am thinking I sound like an idiot trying to seem smart, but I've seen droves of developers who don't even consider this. Businesses who are not even aware of this as a possibility. "We have testing teams to make sure the application is working end to end, we don't need unit testing" or "We have end to end automated testing. For each new feature we write new tests". When you hear this, fight it. Their tests, even if written correctly and maintained perfectly, will take as long as one needs to get on a bus and go to the market. And then the other test will take as long as one need to get on a train and get to the airport. And so on. End to end testing should exist and if you can automate it, great, but it should be sparse, it should function like an occasional audit, not something that supports your confidence in the solution working correctly.

  So go for testable, not for tests. Tests often get a bad wrap because someone like me comes and teaches a company to write tests, then they leave and the people in the company either skip testing occasionally or they change bits of the application and don't bother to update the tests. This leads to my next point: clean code.

Cleanliness

  Cleanliness applies to everything, again. The flow of your solution (note that I am being as general as possible) needs to be as clear as possible, at every level. In software this usually translates in readable code and builds up from that. Someone looking at the code should be able to instantly and easily understand what it does. Some junior developers want to write their code as efficient as possible. They just read somewhere that this method is faster than the other method and want to put that in code. But it boils down to a cost analysis: if they shave one second off a process you run ten times a day, they save one hour per year; if another developer has to spend more than one hour to understand what the code does, the gain means nothing.

  Code should be readable before being fast. Comments in code should document decisions, not explain what is going on. Comments should display information from another level than the code's. Component names, object names, method names, variable names should be self explanatory. Configuration structures, property names, property values, they should be intuitive and discoverable.

  And there is another aspect to cleanliness. Most development environments have some automated checks for your code. You can add more and even make your own. This results in lists of errors, warnings and notifications. On a flow level, this might translate to people complaining about various things, some in key positions, some not. Unit tests, once you have them, might be passing or failing. It is important to clean that shit up! Do not ignore warnings or even notifications. You think a warning is wrong, find a way to make it go away, not by ignoring it, but by replacing the complaining component, marking it specifically in the code as not a valid warning and document why, run all the tests and make sure they are green or remove the tests that you think are not important (this should not happen usually). The reason is simple: in a sea of ignored warnings you will not see the one that matters.

  To be perfectly clear: by clean code I don't mean code that follows design patterns, I don't mean documentation comments on every property and method, I don't mean color coded sections (although that's nice). What I mean is code clean enough to read without cringing or having to look in ten other places to figure out what it does. If your hotdog falls on that code you should be comfortable enough to pick it up and continue eating it.

  Cleanliness should and must be applied to your work process. If the daily meeting is dirty (many people talking about unrelated things) then everybody is wasting time. If the process of finishing a task is not clear, you will have headless chickens instead of professionals trying to complete it. If you have to ask around where to log your hours or who is responsible for a specific job that you need done in order to continue, you need to clean that process. Remove all superfluous things, optimize remaining ones. Remember separation of concerns.

  Cleanliness extends to your project folder structure, your solution structure, your organizational structure. It all has to be intuitive. If you declare a principle, it should inform every query and decision, with no exception. "All software development people are at the fifth floor! Ugh... all except Joe". What if you need Joe? What if you don't know that you need Joe, but you still need him? Favor convention over configuration/documentation, document everything else. And that leads me to the final point: knowledge sharing.

Knowledge Sharing

  To me, knowledge sharing was always natural. In small companies there was always "that guy" who would know everything and couldn't work at all because people came to ask him things. In medium companies there was always some sort of documentation of decisions and project details. In large companies there were platforms like Confluence where people would share structured information, like the complete description of tasks: what they are about, how decisions were made, who is responsible for what, how they were split into specific technical tasks, what problems arose, what the solutions were, etc. And there were always your peers that you could connect to and casually talk about your day.

  Imagine my surprise to find myself working in places where you don't know what anyone else is doing, where you don't know what something is and what it is supposed to do, there are no guidelines outside random and out of date Powerpoint files, where I am alone with no team, brought in for problems that need strong decisions in order to fix but no one is willing to make them, and already I have no idea who should even attempt to. I solve a common problem, I want to share the solution, there is no place to do that. People are not even in the same building as me. Emails are come and go and no one has time to read them.

  Knowledge should live freely in your company. You should be able to search for anything and find it, be able to understand it, contribute to it, add more stuff. It should be more natural for the people in your company to write a blog post than go for coffee and complain. It should be easier to find and consume information from people that left the company than to get it from colleagues at the desk next to you. And maybe this cannot be generalized to all departments, but it is fucking important: people in the office should never need to open Microsoft Office (or any similar product suite). I can't stress that enough.

  You should not need printed documents, so no need for Word. Excel files are great for simple data tasks, but they are not specific. If you need something done repeatedly and you use Excel sheet, it is probably better to build a tool for it. Don't reinvent the wheel now, but use the best tool for the job. And there are better and more modern tools than Powerpoint files, but I will allow the use of them because, in the context of knowledge sharing, everyone should feel free and confident enough to make presentation for the team. My tenet still stands, though: the Powerpoint file would be used in a presentation. Hardly anyone else should need to open it. I mean, there would be a video of the presentation available, right?

Vision

  Imagine a park. It is sunny, birds are singing, there are people walking on hardened dirt walkways, cyclers biking on their asphalted bike lanes, benches everywhere, with a small notepad attached to them that people can just pick up and read or write their own notes. Places in the park are clearly indicated with helpful arrows: children playground, hotdog stand, toilet, football field, bar, ice ring. Everything is clean, everybody is doing what they do best, all is good. You feel hungry, you see the arrow pointing towards the hotdog stand, you walk there calmly and ask for one. The boy there give you a bun and a wurst. He is new, but he has a colleague that knows exactly how much mustard and ketchup to put on the hotdog. He even asks you if you want curry on it. 

  Imagine a park. It is sunny, birds are singing. Some walkways start of as asphalt, then continue as dirt. Some stop suddenly or end in a ditch. There is a place that serves hotdogs next to a toilet. You have to ask around to find out where to find it. You get lost several times, as some people don't know either, but they still come with an opinion, or they are just misinformed. You get tired, but you can't sit on a bench, they are all taken and there are so few of them. You have to look both ways several times before you walk to the stand, because of cyclers. You stand in a line, then order a hotdog. The boy there gives you a bun with a wurst in it. You ask for mustard, but the boy is new and it takes him a while to find it after looking for some paper that tells him where it is. You have to dodge a football that was coming at your head. Someone flushes the toilet.

  I've accepted the old man should teach me as the only solution to becoming a champion, but it is hard to swallow it. He is very old, but mischievous, so whenever I try to learn something from him, he kicks me to the ground. He tricks me again and again and again. I am frustrated, but I am trying to keep my cool. I am strong. If I were to really fight him, he might be smart, but every attack would break bone and then what good would he be? Just a bag of meat and broken shards. I close my eyes, I breath, I tell myself it is worth it.

  The old man apologizes and offers me a hand, I take it, only to be kicked in the ass and thrown into a jumble of debris. I lose my temper and stomp away. He doesn't understand. Getting angry at him is pointless, hurting him futile. I have nothing to learn from him. I walk through the old grounds of my conquests, now just the walled in and decrepit underground of the large arena above. I feel a presence behind me and I see the old man is following me, eyes to the ground. Contrition? Surely another of his tricks. "Begone!" I roar at him, but he goes to his knees and kowtows in front of me, his hands touching my feet. I feel tears swelling up in my eyes. He might as well be a little boy asking for forgiveness. Just who is the teacher and who is the student? Who is the adult here?

  "How did you get to a hundred years or whatever behaving like a little kid?! You are a child!" I shout at him in admonishment. I look around and ghosts of my past awaken my anguish. I feel my face contort into a painful grin as my tears flow freely. "Every week I was coming here to murder people!", I rage, my voice barely my own, a booming, low, animal growl, my expression that of an enraptured madman, for sure. "I would stake my life every time and I would leave, alive, every time!". The images of old fights flash before my wet blurred vision and I imagine that some of the painted white walls might contain some of the scrolls of the ancient arts, built over by a world that doesn't get it anymore. "I loved it!", I say, walking in the dead halls, every step a pulse of power overlaying glorious past over grey reality. My body is shaking with now uncontrollable weeping. "I killed so many people and I miss it... so.... very... MUCH!".

  Does he get it now, I ask myself? Has he even an inkling of the power he needs to teach me to control? I burst through the door to the surface and climb the stairs that get me to the arena above. The seats are packed with oblivious spectators, all watching some performance I don't even care to notice. I breathe in the fresh air and feel better. Ready to come to a final understanding with the old man, if he is capable of it ,I turn around. There is little time and we should not fight each other. But the old man is gone.

   I strain my eyes into the darkness of the stairs and I feel it, The Beast, the adversary I need to fight is there. He's got the old man and, even if I cannot see it, I know it is there, all cunning, fury and power. My body roars by itself, a predator sound, strong and fearless, no sound a man should ever be able to make. The arena spectators panic in surprised horror, but I ignore them. I jump into the darkness with animal strength. I will fight this beast, I will meet it head on, I will be the most savage, alone I will remain alive.

  Sometimes a good regular expression can save a lot of time and lead to a robust, yet flexible system that works very efficiently in terms of performance. It may feel like having superpowers. I don't remember when exactly I've decided they were useful, but it happened during my PHP period, when the Linux command line system and its multitude of tools made using regular expressions a pretty obvious decision. Fast forward (waaaay forward) and now I am a .NET developer, spoiled by the likes of Visual Studio and the next-next approach to solving everything, yet I still use regular expressions, well... regularly, sometimes even when I shouldn't. The syntax is concise, flexible and easy to use most of the time.

  And yet I see many senior developers avoiding regular expressions like they were the plague. Why is that? In this post I will argue that the syntax makes a regular pattern be everything developers hate: unreadable and hard to maintain. Having many of the characters common in both XML and JSON have special meaning doesn't help either. And even when bent on learning them, having multiple flavors depending on the operating system, language and even the particular tool you use makes it difficult. However, using small incremental steps to get the job done and occasionally look for references to less used features is usually super easy, barely an inconvenience. As for using them in your code, new language features and a few tricks can solve most problems one has with regular expressions.

 The Syntax Issue

The most common scenario for the use of regular expressions is wanting to search, search and replace or validate strings and you almost always start with a bunch of strings that you want to match. For example, you want to match both dates in the format 2020-01-21 and 21/01/2020. Immediately there are problems:

  • do you need to escape the slashes?
  • if you match the digits, are you going for two digit month and day segments or do you also accept something like 21/1/2020?
  • is there the possibility of having strings in your input that look like 321/01/20201, in which case you will still match 21/01/2020, but it's clearly not a date?
  • do you need to validate stuff like months being between 1-12 and days between 1-31? Or worse, validate stuff like 30 February?

But all of these questions, as valid as they are, can be boiled down to one: given my input, is my regular expression matching what I want it to match? And with that mindset, all you have to do is get a representative input and test your regular expression in a tester tool. There are many out there, free, online, all you have to do is open a web site and you are done. My favourite is RegexStorm, because it tests .NET style regex, but a simple Google search will find many and varied tools for reading and writing and testing regular expressions.

The syntax does present several problems that you will hit every time:

  • you cannot reuse parts of the pattern
    • in the example above, even if you have clearly three items that you look for - year, month, day - you will need to copy/paste the pattern for each variation you are looking for
  • checking the same part of the input string multiple times is not what regular expressions were created for and even those that support various methods to do that do it in a hackish way
    • example: find a any date in several formats, but not the ones that are in March or in 2017
    • look behind and look ahead expressions are usually used for scenarios like this, but they are not easy to use and reduce a lot of the efficiency of the algorithm
  • classic regular expression syntax doesn't support named groups, meaning you often need to find and maintain the correct index for the match
    • what index does one of the capturing groups have?
    • if you change something, how do other indexes change?
    • how do you count groups inside groups? Do they even count if they match nothing?
  • the "flags" indicating how the regular engine should interpret the pattern are different in every implementation
    • /x/g will look for all x characters in the string in Javascript
    • C# doesn't even need a global flag and the flags themselves, like CaseInsensitive, are .NET enums in code, not part of the regular pattern string
  • from the same class of issues as the two above (inconsistent implementation), many a tool uses a regular expression syntax that is particular to it
    • a glaring example is Visual Studio, which does not use a fully compatible .NET syntax

  The Escape Issue

The starting point for writing any regular expression has to be a regex escaped sample string that you want to match. Escaping means telling the regular expression engine that characters from your sample string that have meaning to it are just simple characters. Example 12.23.34.56, which is an IP address, if used exactly like that as a regular pattern, will match 12a23b34c56, because the dot is a catch all special character for regular expressions. The pattern working as expected would be 12\.23\.34\.56. Escaping brings several severe problems:

  • it makes the pattern less humanly readable
    • think of a phrase in which all white space has been replaced with \s+ to make it more robust (it\s+makes\s+the\s+pattern\s+less\s+humanly\s+readable)
  • you only escape with a backslash in every flavor of regular expressions, but the characters that are considered special are different depending on the specific implementation
  • many characters found in very widely used data formats like XML and JSON are special characters in regular expressions and the escaping character for regex is a special character in JSON and also string in many popular programming languages, forcing you to "double escape", which magnifies the problem
    • this is often an annoyance when you try to store regular patterns in config files

  Readability and Maintainability

Perhaps the biggest issue with regular expressions is that they are hard to maintain. Have you ever tried to understand what a complex regular expression does? The author of the code started with some input data and clear objectives of what they wanted to match, went to regular expression testers, found what worked, then dumped a big string in the code or configuration file. Now you have neither the input data or the things that they wanted matched. You have to decompile the regular pattern in your head and try to divine what it was trying to do. Even when you manage to do that, how often do developers redo the testing step so they verify the changes in a regular expressions do what was intended?

Combine this with the escape issue and the duplication of subpatterns issue and you get every developer's nightmare: a piece of code they can't understand and they are afraid to touch, one that is clearly breaking every tenet of their religion, like Don't Repeat Yourself or Keep It Simple Silly, but they can't change. It's like an itch they can't scratch. The usual solution for code like that is to unit test it, but regular expression unit tests are really really ugly:

  • they contain a lot of text variables, on many lines
  • they seem to test the functionality of the regular expression engine, not that of the regular expression pattern itself
  • usually regular expressions are used internally, they are not exposed outside a class, making it difficult to test by themselves

  Risk

Last, but not least, regular expressions can work poorly in some specific situations and people don't want to learn the very abstract computer science concepts behind regular expression engines in order to determine how to solve them.

  • typical example is lazy modifiers (.*? instead of .*) which tell the engine to not be greedy (get the least, not the most)
    • ex: for input "ineffective" the regular expression .*n will work a lot worse than .*?n, because it will first match the entire word, then see it doesn't end with n, then backtrack until it gets to "in" which it finally matches. The other syntax just stops immediately as it finds the n.
  • another typical example is people trying to find the HTML tag that has a an attribute and they do something like \<.*href=\"something\".*\/\> and what it matches is the entire HTML document up to a href attribute and the end of the last atomic tag in the document.
  • the golden hammer strikes again
    • people start with a simple regular expression in the proof of concept, they put it in a config file, then for the real life application they continuously tweak the configured pattern instead of trying to redesign the code, until they get to some horrible monstrosity
    • a regex in an online library that uses look aheads and look behinds solves the immediate problem you have, so you copy paste it in the code and forget about it. Then the production app has serious performance problems. 

  Solutions

  There are two major contexts in which to look for solutions. One is the search/replace situation in a tool, like a text editor. In this case you cannot play with code. The most you can hope for is that you will find a tester online that supports the exact syntax for regular expressions of the tool you are in. A social solution would be to throw shade on lazy developers that think only certain bits of regular expressions should be supported and implemented and then only in one particular flavor that they liked when they were children.

  The second provides more flexibility: you are writing the code and you want to use the power of regular expressions without sacrificing readability, testability and maintainability. Here are some possible solutions:

  • start with simple stuff and learn as you go
    • the overwhelming majority of the time you need only the very basic features of regular expressions, the rest you can look up when you need them
    • if the regular expression becomes too complex it is an indication that maybe it's not the best approach
  • store subpatterns in constants than you can then reuse using templated strings
    • ex: var yearPattern = @"(?<year>\d{4})"; var datePattern = $@"\b(?:{yearPattern}-(?<month>\d{2})-(?<day>\d{2})|(?<month>\d{2})\/(?<day>\d{2})\/{yearPattern})\b";
    • the example above only stores the year in another variable, but you can store the two different formats, the day, the month, etc
    • in the end your code might look more like the Backus-Naur (BNF) syntax, in which every separate component is described separately
  • use verbatim strings to make the patterns more readable by avoiding double escaping
    • in C# use @"\d+" not "\\d+"
    • in Javascript they are called template literals and use backticks instead of quotes, but they have the same mechanism for escaping characters as normal strings, so they are not a solution
  • use simple regular expressions or, if not, abstract their use
    • a good solution is using a fluent interface (check this discussion out) that allows the expressivity of human language for something that ends up being a regular expression
    • no, I am not advocating creating your own regular expression syntax... I just think someone probably already did it and you just have to find the right one for you :)
  • look for people who have solved the same problem with regular expression and you don't have to rewrite the wheel
  • always test your regular expressions on valid data and pay attention to the time it took to get the matches
  • double check any use of the string methods like IndexOf, StartsWith, Contains, even Substring. Could you use regular expressions?
    • note that you cannot really chain these methods. Replacing a regular expression like ^http[s]?:// with methods always involves several blocks of code and the introduction of cumbersome constants:
      if (text.StartsWith("http")) {
        // 4 works now, but when you change the string above, it stops working
        // you save "http" to a constant and then use .Length, you write yet another line of code
        text=text.Substring(4);
      } else {
        return false;
      }
      if (text.StartsWith("s")) {
        text=text.Substring(1);
      }
      return text.StartsWith("://");
      
      // this can be easily refactored to
      return text
               .IfStartsWith("http")
               .IfStartsWithOrIgnore("s")
               .IfStartsWith("://");
      // but you have to write your own helper methods
      // and you can't save the result in a config file​

  Conclusion

Regular expressions look daunting. Anyone not familiar with the subject will get scared by trying to read a regular expression. Yet most regular expression patterns in use are very simple. No one actually knows by heart the entire feature set of regular expressions: they don't need to. Language features and online tools can help tremendously to make regex readable, maintainable and even testable. Regular expressions shine for partial input validation and efficient string search and replace operations and can be easily stored in configuration files or data stores. When regular expressions become too complex and hard to write, it might be a sign you need to redesign your feature. Often you do not need to rewrite a regular expression, as many libraries with patterns to solve most common problems already exist.

  I see a lot of pages about how to write blog posts. I read them, because I am both curious and sincere in my desire to make my blog popular and spread the knowledge I've amassed here. They are always crap. Take one that says the best tool to get a blog popular is to use Google Trends or Google autocomplete to see what people are searching for. And the results are always incredibly stupid. Like "how to add one to one to get two". I am paraphrasing a bit here, but you get the gist. Go "worldwide" and the first trend is always some Chinese spam. Another post is saying that a blog post should be written as four drafts: one for what you want to say, one for how you want to say it, one for peer reviewed content and the final one that actually is what you want to publish. It sounds great, but it implies a level of work that sometimes is prohibitive related to the subject of your post. Sometimes you just want to share something as a stream of consciousness and be done with it. Is that better? No. But it sure beats NOT writing anything. There is always time to improve your work and get peer review AFTER publishing it.

  There are two big types of people blogging. The first category is akin to reporters and media people. They want to get their message across for reasons that are rather independent of the message itself. They want to earn money or influence or some other kind of benefit. I don't have any advice for people like that. Not because I disconsider their goals, but because I have never blogged for an ulterior reason. The second category of bloggers is akin to writers: they want to get their message across because they feel there is some value in the message itself. I consider myself such a person, although I probably suck as a writer. This post is for people like that.

  The most important part of writing a post is motivation. And I don't mean just the reason for writing it, but the reason for wanting to share it. For me, most of the posts I write are either content that I consume, such as books, or stuff that I think is worth considering or technical stuff that I've stumbled upon and I believe people would want to find if googling for it instead of wasting the time I wasted to solve it. Now, the books and the personal idea posts I totally agree are ego boosting shit: I feel like it's important enough to "share", but I don't really expect people to read it or that there is any inherent value in them other than getting to know me better. And everyone wants to understand other people better on the Internet, right? In the end they are just a personal log of random thoughts I have. My blog is something that represents me and so I feel that I need to share things that are personal to me, including thoughts that are not politically correct or even correct in any possible way. One can use Facebook for this, so I won't write about those posts. They still reach some people and inform their choices, which is something I love.

  What is left is the posts that I work for. You have no idea how much I work on some of these posts. It may take me hours or even days for content that you read in a few minutes. That is because I am testing my ideas in code and creating experiments to validate my beliefs and doing research on how other people did it. A lot of the times I learn a lot from writing these posts. I start with the expectation that I know what I am talking about only to find out that I was wrong. The important part is that I do correct myself and some of the blog posts here are exclusively about discovering how wrong I was. There is nothing more rewarding than writing something that you feel others might benefit from. Perhaps other than getting feedback about how your post benefited others. Publishing your failures is just as important as publishing your successes.

  Yes, I know, if I learn something new by doing what I need to be doing, then sharing the results is like writing for myself, too. It's ego boosting, for sure. However, it would be even more obnoxious to believe no one is like me and so no one would benefit from the same work. There was a time when people came to my blog and asked me about all kinds of technical problems and I worked with them to solve them. There were usually incredibly simple problems that posed difficulties only to the laziest people, but it felt good! Then StackOverflow came along and no one actually interacts with me. But I have solved stupid problems that I still keep getting thanks for, even (maybe especially because) if the technology is really old and obsolete. Many other blogs published cool things about subjects that are not fashionable anymore and then just disappeared. The value of your content is that it may help people in your situation, even if they don't share your sense of now and even if all they take away is how NOT to do things.

  Sometimes you are looking for the solution for a problem and after hours of work you realize the problem was invalid or the solution was deceptively simple. It's the "Oh, I am so stupid!" moment that makes a lot of people shy away from writing about it. I find that these moments are especially important, because other people will surely make the same mistake and be hungry about finding the answer. OK, you admit to the world you were stupid, but you also help so many other people that would waste time and effort and feel as stupid as you if not for writing the post.

  My take on writing a blog post is that you just have to care about what you are writing. You may not be the best writer out there, you might not even be explaining the thing right, but if you care about what you are writing, then you will make the effort of getting it right eventually. Write what you think and, if you are lucky, people will give you reasons to doubt your results or improve them. Most likely people will NOT care as much about the subject as you, but you are not writing because of them, you are writing for them. Some of your thoughts and toils will reach and help someone and that is what blogging is all about.

  The last thing I want to mention is maintenance. Your work is valid when you write it, but may become obsolete later on. You need to make the effort to update the content, not by removing the posts or deleting their content, but by making clear things have changed, how they did and what can be done about it. It is amazing how many recent posts are reached only because I mentioned them in an "obsolete" post. People search for obsolete content, find out it's too old, then follow the link to your latest solution for that problem. It makes for good reading and even better understanding of how things got to the current point.

  So, bottom line: publish only what you care about and care about your readers, keep the posts up to date, publish both successes and failures.

  Imagine you are playing a computer game, exploring virtual realms and testing your mettle in cooperation or opposition to other players. You are not the best, but you are getting better and you feel that reward system in your brain getting activated and giving you that pleasant buzz, like you are doing something that matters. Suddenly, new players enter the game and they seem indomitable. You can't possibly defeat them: they are faster, incredibly so, they are more accurate, dubiously so, and they seem to have no respect at all for the spirit of the game that you, until just now, enjoyed. They don't want to get better, they want to humiliate you and the other players by just being incredibly better than all with no credible cause other than, yes, they cheat. Somehow they found a way to skirt the rules and thus make them meaningless.

  While this is a scourge that affects all online games, it is also a powerful metaphor about real life. Think about trying to advance in your company, get that job that gives you more money, more prestige and proves to yourself and others that you have become better, a worthy person for that role. Suddenly, a new player arrives, and he is the nephew of the CEO and he gets the job for no credible reason. That is not a game, it's your life. The resentment is real. You can't just change servers or turn off the computer and read a book: this affects you, your family, your loved ones.

  But I will go ever further. Imagine that you are trying to lead a good life, according to the moral principles that were instilled in you by family, culture and your own evolution as a human being. You take care of your children and make efforts to set up their lives so that they have the many and good opportunities. You paid your life insurance, prepared your pension fund and are content that in a decade or so you will finish paying the rates for the house where you plan to retire and live out your golden years. You've taken care of your health, you eat and drink responsibly, you exercise regularly. Suddenly, new players arrive. They have found a way to cheat death. Not only do they have better health, they don't age. They might even get younger and fitter and smarter with no effort at all. Your pension funds implode, because old age becomes irrelevant, the prices skyrocket because there are more people with more money buying stuff and not getting any older and weaker as they go. Your children have no more opportunities, as they can't compete with people that are of the same biological age, but have decades of experience.

  I believe this way of thinking is what instructs most ethical ideas. Life is a game, with rules that are either agreed upon or forced upon the players by the limitations of the environment. Cheating at this game sounds both ideal and amoral. We have a zillion stories warning about the perils of playing god, but in the end they are just a reaction of fear to the mere possibility that someone might find a way to hack life.

  And I agree that it is very dangerous, for the very reasons that game hacking is so annoying: it makes the game irrelevant. If people don't care about life anymore, if they have no limits, then what's the point? It's almost a Nietzschean concept that the worth of life cannot exist in a vacuum, it needs suffering and obstacles to overcome. What would the philosopher believe of someone who becomes better by overcoming hardship only to be completely overshadowed by someone who just ... cheated. What would it mean to live a happy and fulfilling life if you've hacked your brain to feel happy and fulfilled? What would it mean to live a moral life if the ability to disobey rules has been bred out of you?

  Yet, what is the moral ground to not even try, I ask. How can it be moral to conceive of life as just a game? Wouldn't that be meaningless also? I submit that the very possibility of skirting the rules makes them obsolete. I submit that just as talented people are banned from online servers for being too good, talented people are getting sidelined in life by the very same "ethical" way of thinking of life as a static game where people should follow the same rules and achieve the same relative rewards.

  As technology and knowledge and sheer individual power increase, the danger of people playing god is dwarfed by the danger of people killing god inside themselves.

  I see only one solution for this: the expansion of the human race. Only when centralized authority becomes impossible will humanity truly reach its potential. That requires we spread out so far that enforcement can only be local. It will permit us, if you will, to have different servers to play on. Some of them will ban cheaters, some of them will welcome them, and there will be many variations in between. Some of them will try and fail, maybe spectacularly, but some of them will thrive and advance not only the players, but the game itself.

Just has a revelation. There are studies that show the moment you introduce currency in a social transaction, the dynamics change dramatically, leading to conflict, selfishness and the dissolution of societal and even emotional bonds. For a random reference check this article: Why Good Deeds and Money Don’t Mix.

I've been struggling with this new political correctness movement because 1) I didn't get it 2) almost every one of the people actively acting offended in this context appears to be... not nice and 3) it doesn't seem to be helping any. So, am I the bad guy? I started to ask myself. Am I a racist homophobic sexist misogynistic normie white male working in the tech field or is there something else going on? Judging by how far from normie people who actually know me think I am, I started to think about it more.

And it came to me! Political correctness is a form of currency forcefully introduced into our social transactions. Not only is it causing trouble for people who are assholes, but also for normal people who suddenly feel they have to pay something. And, as currency does, it breaks society, not strengthens it.

That is why so many people caught in this are so violent and partisan about it. That is why when you are nice towards a - I don't even know how to call them these days - not white person it feels good, as it would being nice towards anybody else, but when you are forced to do it, it well... feels forced. It feels like duty, like work, like paying a tax. The concept of balance slowly creeps in and makes one push back. Maybe with a joke, maybe with an angry tweet, maybe with something worse like actually picking on someone for their skin color, sex, age, religion or anything else. And they do it because picking on someone for being... I don't know... Romanian, doesn't feel like restoring anything. And now Romanians are pretty angry, because offending Jewish people or of recent African descent is somehow "wronger", so they get offended and feel left out. It's wrong to pick on anyone either way, deal with it!

In the end, introducing currency just pushes people into two diametrical opposed groups: the payers and the people who are owed. And of course, the people who ride the wave and get their little percentage to convert it to any other currency: money, hate, power, etc. We become slaves to the middlemen even when we interact with other people! Hell, they want to introduce ethics for computers now. Where does it end?!

So, as I am an egalitarian in my misanthropy, I submit that you should get offended by people just like any other person would. Leave currency to bankers. Or pick on them! No one ever got into a twist for calling bankers names.

There are few big questions as big as "Why death?" and "Why getting old?" and "Why sex?. And it's surprising that the answer is probably parasites.

After a lifetime of fighting disease and random damage, it becomes more efficient to give up and just start anew. Death comes just as a natural consequence of a body that has (or should have) achieved replication already and is out of warranty, so to speak.

Sex is similar. Once a parasite finds a way in, it makes a whole lot of other parasites that know that way, too. If an entire population is the same, clones of the same splitting creature, they are all just as vulnerable. Sexual reproduction mixes things up and leads to more variety, jump starts evolution and maximizes the chances that a portion of a population is immune to any one attack.

And I've been thinking today, as a tangent, that another big question might have the same answer: "Why are there sociopaths?". If societal development and cooperation is the solution for gaining supremacy over individualistic populations, why aren't we all social and empathetic and nice? And I am thinking: empathetic people want to help others, especially sick and hurting people. Instead, they get sick themselves and help spread the disease. It helps to have a small, but significant minority, of assholes who only care about themselves. In case of a major outbreak, they stay clear and survive.

I am going to try something new with this blog post. Usually, when I go somewhere on vacation, the things that capture my attention are not what interest about anybody else in my entourage. I am also not a very lyrical writer, so what's the point of enumerating the places I've been from the perspective (oh, so much used!) of the casual tourist. I imagine myself writing one of those horrid "10 things to do in..." articles, promptly vomit and desist from thinking about it.

With this post, though, I am going to tell you of the wild (but accessible) area that I've explored and where to find it, how I felt and, if I can find the references, what plants and animals live there. You see, when I go somewhere, I avoid people and take really bad pictures of flowers and plants, butterflies, weird things and sometimes landscapes.

The place


I've been to Cheile Bistritei Valcene (the canyon of the river Bistritza from Valcea - there is another one in the north of the country) and in the valley of the Luncavatz river. The vegetation and insects are very similar, so I am going to treat this as a single area, even if their locations are 20 KMs apart:
  • Cheile Bistritei: from 45.189828, 24.039859 to 45.197871, 24.030284
  • Raul Luncavat: from 45.186682, 23.917856 to 45.190084, 23.914488

The area is very nice, easy to get to by car, but not very touristic yet, so not a lot of people having picnics and listening loudly to music. Leave your car and walk on the sides of the river(s) and the scenery is verdant and quiet. I have to warn you that even if in Romanian they are called rivers, they are more like creeks, especially at this time of the year. You can even find some caves in the area and if you are the long walk type of person, some 4-5 hour hike routes to more remote areas. Some flowers are white, but the vast majority of them are either yellow-orange or violet in color and probably are much more interesting in ultraviolet than human vision.

Animal life


I haven't seen any animals other than birds, a running lizard and a lot of insects.

I saw several species of butterflies, the most common by far being a medium sized orange with black spots, a fritillary, probably the Silver-washed fritillary (Argynnis Paphia) or mantia imparatului in Romanian. They were frolicking on these tall yellow flowers with large leaves: the yellow oxeye (Telekia speciosa) or brusture and ochiul-boului in Romanian. The next most common was a shy dark butterfly with a crimson edge on its wings. Probably the Woodland ringlet (Erebia Medusa), I have no idea what the Romanian popular name for it is.


Argynnis Paphia on a Telekia speciosa




Erebia Medusa



Some other butterflies: the cabbage white (Pieris rapae) or fluturele de varza in Romania, the peacock (Aglais io) or ochi de paun de zi in Romanian, the swallowtail (Papilio machaon) or coada de rindunica in Romanian and even one glimpse of what I think was a marbled white (Melanargia galathea) or tabla de sah in Romanian.


Pieris Rapae




Aglais Io (during my youth Innachis Io)




Papilio Machaon




Melanargia Galathea



One fascinating specimen looked similar to a peacock butterfly from afar only for it to settle in a triangular black and white shape when it stopped. It must have been a moth! I've identified it as a Jersey tiger moth (Euplagia quadripunctaria) or fluturele urs dungat in Romanian.


Euplagia Quadripunctaria



What I also found fascinating is a tree that had apparently been colonized by ants. A lot of wood dust was on the ground and a lot of activity was inside the trunk. Still, there was another hole in the trunk that was filled with wood dust, but no ant activity. Could it have been some sort of other factor, termites or perhaps a disease, that destroyed the tree trunk's interior and the ants were just opportunists?


Ants or termites? New behavior or opportunism?



Plants


Now, plants are easier to photograph, but harder to identify. I've mentioned the oxeye. Then there was the touch-me-not (Impatiens noli-tangere) or slabanog and bradulet in Romanian, which appears to have been used traditionally for its medicinal properties, mostly related to kidney or gynecological issues. The Spreading bellflower (Campanula patula) was there, together with its close relative, the Creeping bellflower (Campanula rapunculoides) - both are called clopotei in Romanian. There was the mountain geranium (Geranium robertianum) or napraznic and priboi and iarba sfintului Robert in Romanian, which appears to have anti-stress, anti-cancer and fertility related purposed in traditional medicine.


Impatiens Noli-tangere




Campanula Patula




Campanula Rapunculoides




Geranium Robertianum



Field mustard was present as well (Brassica rapa). This plant was and is used for a variety of reasons in many cultures. The leaves and roots are rich in oil. In Puglia they use the buds as cimedirape in the making of orecchiette pasta. It is a plant from the cabbage family of plants, probably explaining the presence of the cabbage butterflies.


Brassica Rapa



A very interesting flower has a very weird name: the Viper's bugloss (Echium vulgare) or iarba sarpelui in Romanian. It has blue flowers but the filaments of the stamens are red, contrasting with the petals and giving it a violet color per whole. It has medicinal uses as well, as an antidote for snake bites and for its antibiotic and astringent properties.


Echium Vulgare



Another violet flower, with uses in homeopathic medicine but also in poisonings, is aconite or wolfsbane (Aconitum napellus) or omag in Romanian. It contains powerful alkaloids and at one time it was forbidden to grow this plant anywhere in the Roman Empire on penalty of death. Death from intoxication with the plant can occur in as little as half an hour!


Aconitum Napellus



And since we are talking about a violet flower with medicinal properties, how can we ignore the heal-all (Prunella vulgaris) or busuioc salbatic in Romania. The young leaves and stems can be eaten raw in salads; the plant in whole can be boiled and eaten as a potherb; and the aerial parts of the plant can be powdered and brewed in a cold infusion to make a beverage and it is used as an astringent in folk medicine.


Prunella Vulgaris



Not often, but when it happened it was a whole field of them, I found the Orange mullein (Verbascum phlomoides) or luminarica in Romanian. It is also a medicinal plant, used for the calming, sweat inducing and expectorant effects.


Verbascum Phlomoides



I've seen some daisies in the area and also another flower from the same family: the fleabane (Erigeron annuus) or bunghisor in Romanian. Used in salads as well as against the common cold or stomach aches in folk medicine.


Erigeron Annuus



Last, but not least, the evening-primrose or sundrop (Oenothera biennis) or luminita noptii in Romanian. It started as an American plant, much like the fleabane, but it was brought and naturalized in Europe. It has been used medicinally by the native Americans for all kinds of ailments, as it is an edible plant containing an oil with anti inflammatory properties.


Oenothera Biennis



There were a lot of plants without flowers, but I haven't had the time or patience for them. There was one with huge leaves and I photographed it for identification purposes. It turns out it was either the butterbur (Petasites hybridus) or the burdock (Arctium lappa) both called brusture in Romanian, but different species altogether. It's probably the Arctium, but I can't be sure!


Petasites hybridus or Arctium lappa? A lot of stuff called brusture in Romanian!



Resources


It would have been a lot more difficult for me to write this post if it weren't for sites like:
Even with these great resources, it was obvious that not many people will publish nature related posts in any systematic manner. Even this post, three hours in the making, is a random mess of blurry pictures and random observations.

There is this old joke about how having a senior developer around is like having a little child sitting next to you. "How do I do that?", you ask. And the SD asks "Why?". "Well, the boss asked me to" "But, why?. "The client wants it" "Why?". "Because he needs this other thing" "Why?". I am afraid the joke is very true. Usually, at the end of the exchange you get to the real need at the base of the request and understand that it is the thing that must be fulfilled, not the request as it came through several filters, each adding or removing from the real purpose of the task. And if there is something I want to impart from my extensive experience as a software developer it is that you must always start from the core need and go from there.

However, I am not here to discuss how to write code, for once, but on how to organize your team (or yourself as a team of one) from this need driven perspective. And please don't call it NDD and read it NEED or whatever, because this is just a general principle that can help you in many domains. So let's analyse Agile. It's been done to death, so what I am trying to do here is not summarize what others have done, but to simplify things until they don't even need a name anymore.

First, what is Agile Development? Wikipedia says: "Agile software development is an approach to software development under which requirements and solutions evolve through the collaborative effort of self-organizing and cross-functional teams and their customer(s)/end user(s). It advocates adaptive planning, evolutionary development, empirical knowledge, and continual improvement, and it encourages rapid and flexible response to change. The term Agile was popularized, in this context, by the Manifesto for Agile Software Development. The values and principles espoused in this manifesto were derived from and underpin a broad range of software development frameworks, including Scrum and Kanban."

So Agile is not Scrum, but Scrum is underpinned by Agile principles and values. That is important, because many shops want to do Agile and so they do Scrum or Kanban and they either try to follow them to the letter (purists) or adapt them to their requirements (which is more in line with Agile principles, but the implementations usually suffer). One wonders how come there are not a lot of different methodologies out there, just like there are software patterns or programming languages. The answer is that no one starts off with understanding their need for Agile and in fact many don't even know or care what it is. They do Scrum so that they get the Agile badge, they tell everyone they "do Agile" and pompously ask their candidates if they "know Agile".

But if someone asks "Why do you want to do Agile?" the answers are not so easy to get. Of course there are advantages in the approach, that is why it's so popular, but you don't get advantages for your collection, they need to drive you to some kind of goal. One might say that for every process you need metrics in order to analyse your progress. Scrum does that. You don't need metrics for the efficacy of the metric system, do you? But still you need some way of gauging the usefulness of your methodology itself, otherwise we would still be using imperial units.

I am more accustomed to Scrum, but I don't want to analyse it, instead I want to measure the usefulness of Agile as a concept by asking the very simple question "Why?". As we've seen, its attributes include:
  • self organizing teams
  • cross functional teams
  • collaborative effort between teams and customers
  • adaptive planning
  • evolutionary development
  • empirical knowledge
  • continual improvement
  • rapid and flexible response to change

Why self organizing teams? That's the first and one of the most important and complex aspects of Agile. If the team self organizes, then everyone in it shares responsibility. There is no need for micromanagement from above, the team acts as a unit, you get it as a neat package that you don't care about. In theory when a team is inefficient from the organization's point of view, you should fire the team, not individual people. If one member is not performing as expected, then it's the responsibility of the team to fix them or get rid of them. Also theoretically, an Agile team doesn't need a manager.

But in practice I haven't seen this implemented except in small startup teams. There is always a manager, a director or more of them. There is always micromanagement. There is always some tendril of HR or organizational graph that affects people individually. Why is that? Because teams are cross functional, too. A developer doesn't need or want in any way to have to measure the work efficiency of their colleagues, have to tell them how to do things or get to the point where they have to push for firing them. A developer doesn't want to administrate the team. So what is a manager in the Agile world? That's right, a member of the team. If devs would not have a manager, they would probably hire one, as part of the self organization of the team. Moreover, for teams inside larger organizations like corporations, the "customer" is the corporation, so the manager role is often combined with the one for "customer representative" or "product owner" or "producer" or something like that. They're the people you go to in order to understand what it is you have to do.

And since I've started talking about cross functional teams, this means that Agile doesn't need or indeed recommend that people be interchangeable, having the same skill sets. Whenever you hear someone say "everyone should know everything" that's not Agile. It's just a sign that someone from above wants to micromanage the team without having to know anyone in it or ever decide the direction it should work in. It's the sign of a top-down approach that is ultimately inefficient from both directions: the team feels pressure it doesn't need while being restricted in what it can do and the management has to perform tasks that they don't need to do. I haven't seen one team of people treated as interchangeable units be efficient, nor have I heard of any. So why cross functional teams? Because people are individual beings and are different and, for every skill they have, they put in effort to first reach a basic level of knowledge, then more effort to become good or even expert at them. An Agile team needs to be flexible enough to accommodate for any type of person, within reason.

As an aside, think of the skills of a challenged person. The legislation of many countries requires now that software be accessible to a wide variety of disabled people. So whenever you hear that people should be versed in all aspects of the team's business ask them for Braille courses.

Cross functional also includes the skill of knowing what the customer wants, whether they are a member of your organization or someone sent by the client. It's one of the most necessary roles to be filled in a team. If a team were a living thing, the product owner would be the eyes and ears. You can be strong, fast, deadly, but it doesn't help if you can't find your prey.

So why collaborative effort between team and customers? Because you need to know why do you what you do. This entire post is dedicated to the question of why, so I won't press this. Just remember that if the team doesn't know why something is required or necessary, then it is not yet fit to perform that task. People are going to hate me for this, but we've already established the need to understand the client and that in large organizations the customer is the organization, so the logical conclusion is clear: office politics, psychopathic displays of power, idiotic decisions coming from up high are all part of the requirements. There is a caveat, though: the need of the customer needs to be clear, not the want. If the liaison to the customer is withholding or unaware of the real reasons something needs to be done, then the team suffers.

As an aside to all manager or high corporate types out there, it's way better to explain to a team that they have to scrap a project because you hate the guts of the person who proposed it, than to invent business reasons that are clearly bullshit for destroying months of loving work. In the first case you honestly admit you are an asshole, but in the second you prove you are one.

Now, what's this about adaptive planning? If you already know what is needed, someone should let you do your job in peace. You come to them when it's done. That was the fallacy of the Waterfall system. That is also why Waterfall still works. In a situation where needs are clear and do not change, there is no need for agility. Unfortunately, these situations are rare, especially since I've included in the list of customer needs the legacy of us being descended from apes. That means adapting quickly, without regret, changing direction, sometimes turning back completely, abandoning projects, starting new ones that you recommended starting years ago and was laughed out from the room for it and so on.

It's not only that. Adapting to your client's need is something that you would have to do anyway, in any realistic scenario. But it also means that you have to adapt to the technological changes in the world. Maybe you have to rewrite your code with another technology or explore new ones that just appeared out of thin air. Adaptation means that a big part of your job as an Agile team is exploration. This is emphasized by the empirical knowledge attribute above. These two go hand in hand. You adapt by understanding and for this you must be in the loop, you need to know things. Well, one of the members of the team needs to, at least. Adapting means "rapid and flexible response to change".

Therefore, whenever someone asks you if you are Agile or demands that you are, you can tell them about all the new tech that is rumored to come out this year and how you can barely contain your enthusiasm to be working for someone who enables exploration in the true spirit of Agile. Preferably after you've signed a contract, maybe.

But I talked about adapting to new requirements and changes in the environment, not adaptive planning. Adaptive planning is planning for all the things above. In other words, when you start a task, plan for the possibility of it changing, having to be abandoned or having to be redone in a different shape. And here is a point of much contention, because you can do this as any level. If you do it whenever you have to change the color of a button, you will overestimate all your work. If you do it at the level of the entire enterprise suite of products, you might as well do Waterfall. In any respect, it is the entire team that needs to know and decide what they plan for drastic changes.

Yet there are two types of reacting to change: planned or chaotic panic. Adaptive planning refers to planning for change, not preparing for working in complete chaos. That is why Scrum, as an example, splits planning into small periods of time (2-4 weeks) called sprints for which the work planned needs to be finished. If anything drastic (like a task losing its relevance) happens, the sprint is aborted. The sprints themselves can be in relative chaos, but inside one planning and order prevail. It doesn't mean planning for an irate boss man to come and tell you every day to change this and that. It means having a list of tasks that you can edit, whether by removing or adding tasks, rearranging their priority, or even adding tasks like "how to deal with this insane man coming into the office and spouting obscenities at me", then execute them in order. In no way does it mean that you do things by the ear or plan for something and then ignore it.

Two aspects remain to be discussed: evolutionary development and continuous improvement. Both strongly imply awareness of the development processes through which the teams goes. It's the team equivalent of introspection commonly referred to as retrospective. What did we do? How does it compare with what we had planned? What does it mean for the future? How can we do better? and the Why of it is obvious, you've got a good thing, how can you get more of it? Yet this is the point that is hardest to put in a box. Various methodologies attempt to introduce metrics to measure progress and performance. Then, with the function of the team properly computed and numbers assigned to its yield, one can scientifically tweak the processes in order to get the most out of the resources the team has. This is a wonderful concept and I agree with it in spirit, but in practice frankly, I think it's bollocks. It pure shit. It's circular logic.

Here we are, discussing a beautiful philosophy of doing work for maximum benefit by letting a team of very diverse people organize themselves towards a common goal, being ready for anything, adapting to everything, pushing through no matter what by leveraging their individual strengths. And then we come and say "Wait, I can take all this, name it, measure it, put it into equations and then tell you how to improve it!". Well, why didn't you do that from the start, then? Why bother with all the self expression and being able to adapt to anything new? If all is old under the sun, why do Agile at all?!

I believe this is the part where Agile methodologies fail utterly and miserably and can't even admit to themselves that they have. I agree that progress needs to be visible and the ability to compare it with other instances is essential, but from here to measuring it numerically is a large distance. Forget the usual arguments against measuring team success: the team composition is not fixed, the type of work it different, team dynamics means people behave differently, personal and local events, and so on. It's bigger and simpler than that. I submit that since the entire concept of Agile rests on the principles of flexibility, the method of gauging success should also be flexible and especially the way the team processes are tweaked for the future.

And I have proof. In every company I've ever been or heard of the Agile methodology of the place didn't quite work in various ways and that was apparent when people were laughing or disrespecting that part to the point of ignoring the rules in place. And the common bit that didn't work for all of them is the retrospective. The idea of asking "what problems did we face?" and then immediately finding and enforcing solutions with "what can we do about it next time?" has become such a mantra that it is impossible to ask these questions separately. It's become reflex (to the horror of wives everywhere) when you hear "there is a problem" to ask immediately "what can we do to fix it?". There is an intermediate question that must be inserted there: "Is this really a problem or do I just have to be aware it happened? Can we ignore it?". This is another topic, though.

A short summary would be:
  • responsibility is shared between all the members of the team and decisions should be taken as a team
  • any member of the team is valued for their contribution towards reaching the team goal
  • the manager is not your boss, it's just the role of a member of the team
  • the Scrum master or Kanban master or whoever does the accounting in whatever methodology you use is still a member of the team
  • having a member of the team understanding the needs of the customer is essential; they are the people you go to ask "Why?" all the time
  • adaptation need to be written in the DNA of the methodology; the team must plan to change its plans, but still do things orderly
  • experience should guide the team to define progress and improve itself
  • transparency of the team goal is necessary for the team's success
  • there is no improvement of the development method (or the code) without exploration and the entire intentional participation of the team
  • never be afraid to ask why; if you work in an environment where you are afraid, consider changing it

Let me tell you the typical story of the development team. First there is chaos, someone decides they are a quick and dirty team and they can do anything, fast, cutting all corners into a straight line to hell. Mistakes are made, blame flies around, people are fired, emotions get high, results go nowhere. There is need for order, all cry out, we need to use the holy grail of software development and the greatest human invention since fire (that week, at least). And then a specialist is called. He either becomes part of the team, with the role of teaching and enforcing rules, or just teaches a few courses, gets their money and leaves before hapless fools try to put in practice whatever it is they thought they understood. The result is the same: developers first like it, then start grumbling about the indecency of having to fill out documentation before any code is written, or updating it, or adding tasks in whatever software is used (or Post-Its on a whiteboard, if you are a purist and still haven't figured out developers can't use a pen to write). So either morale plummets and the operations continue with diminishing results, or the grumbling becomes a rallying cry to make a change. And the change is... chaos again.

In short, just like a revolutionary South American country, they can't decide if they want freedom or dictatorship, because both options have big damn flaws and anything in between is sucking their life away.

But why?

It's because even methodologies are subject to change, to human whim, to altering conditions. If you look at the beginning of this enormous post, it all started with the main goal and meandered from that, just like an Agile methodology in a real firm does. What is the goal of the team? And don't you look at your manager to tell you. As a team, responsibility is shared, remember? If the purpose is to turn perfectly enthusiastic software developers into soulless drones... you should run away from there. But if it is not, think about the things you are doing. Are they helping or are they not helping? This week at least.

In conclusion (or retrospect?) I believe Agile is a great idea, but its underpinning principles of always being aware of the situation and adapting to it are a more general and useful concept to remember and live by. Truth and knowledge comes before adapting and many software shops fail there, so the methodology is irrelevant. Then the method of work chosen should reflect the flexibility it enables. I don't believe in a pure anything, and certainly not pure Scrum, pure Kanban, pure Agile, but whatever you cook up, it should work towards the real goal of the team, as known and acknowledged by everyone in it. What's your function in life?

No natural phenomenon, except maybe fire, seems more alive than the mist. But while fire is young, angry, destructive, mist is an old grumpy creature, moving slowly, hiding itself in contradictions. It doesn't hide distant features as much as it reveals close ones through contrast, it doesn't absorb light as much as it lets itself glow around sources of illumination, it makes sounds crystal clear by covering the constant hum of far off noise, dense and yet immaterial, its blanket like qualities offset by its cold embrace. Never more life like than when it clings to a still surface of water, the slightest gust of wind prompts annoyed tendrils and every move of another living thing elicits mirror acts, dream like, half finished motions forgotten before they even end. If mist could only remember...

It is said that the great theory of relativity of Einstein's doesn't apply to things moving slowly. Today I realized that is not true. There is a direct relationship between space and time and speed affects space, so it must affect time. Here is a practical example: a car moves faster than a person walking, so its speed makes distance shrink relative to time. Inversely, that means that it makes time expand, become more expensive, from the car's point of view.

That is why, when you see a car approaching and you have the option of walking in front of it forcing it to stop, you wait, because the driver's time is more expensive than yours. Stopping the car and wasting time would impact him much more than it would you. It also has the side effect that it saves your life if the car doesn't stop for some reason.

Just a thought.

The Romanian language has a word: deștept. It means smart, but it leans into knowledgeable, so it means both "knowing things" and "thinking fast". There is no relation to wisdom and this is the case in other languages as well. Sometimes wise is used to denote knowledgeable, yet I don't think they are related. While to know things means to be able to recall things you have learned, wisdom, I've come to realize, means to understand what little you know. Someone might be wise and know very little and think rather slowly. Wisdom is the maturation of the soul, like a well kept wine it provides subtle flavors.

Even a superficial and forgetful person as myself can gain wisdom in time. It is important to note this, because as people get older, stuck between that limit of usefulness and the onset of senility, we tend to dismiss them, flaunt our new found (and invented) knowledge to their faces, ignoring a very important aspect of their evolution: wisdom. Sure, their wisdom might not apply to your field or need, but even if it were, are you acknowledging it?

Just a thought.

Siderite's Razor: "The simplest solution/explanation is often somebody whining"