Tuesday, June 6, 2017

A More Probabilistic Approach To Life

Thomas Bayes (Credit: Wikipedia)

I recently read Nate Silver’s The Signal And The Noise (2012). Silver, the founder of statistical analysis website FiveThirtyEight, is arguably the most prominent statistician and forecaster in the United States today.

In this book, Silver seeks to answer a pressing question: Why do so many predictions fail? More specifically, why is it so hard to distinguish the signal (what Silver describes as “the truth”, or what is really important to observe) from the noise (“what distracts us from the truth”)? Silver also offers a solution: a probability-focused, statistical approach to thinking, specifically by applying a formula developed by mathematician Thomas Bayes, to better shape and inform our thinking, in many areas of life.

Silver cites a range of examples, involving both individual and collective judgment, where these issues play out. Most people aren’t very good at playing poker, or betting on athletic events.  Intelligence analysts failed to predict either the Pearl Harbor or 9/11 attacks. Very few investors (professional or others) successfully beat stock market indexes, over the long run. These finance professionals share the company of economists, who largely failed to anticipate the 2008 financial crisis (both in terms of likelihood and severity). In fact, economists have done rather poorly, in terms of forecasting economic growth and recessions , over the past few decades as a whole. A projected projected outbreak of the H1N1 virus in 2009, never materialized. Political prognostication by (largely partisan) pundits, generally produces mediocre results. Silver even dives into the results of efforts to predict natural phenomena, such as earthquakes and inclement weather, exploring why these undertakings often don’t succeed.

What’s going on here? Do we lack the mental hardware, to see into the future? Not neccessarily. Let’s consider some of the reasons why predictions fall short.

During the 2008 financial crisis, homeowners overlooked the possibility of a large drop in home prices, since that hadn’t happened in the recent past, while economists underestimated the impact of housing prices on the financial system. Both these predictions proved incorrect, in large part because prior data was out of sample and thus not applicable: housing prices had never risen quite this much, in such a short time, while the financial system had never been quite this leveraged, with such large amounts of debt.

Political forecasters frequently fall short as well. The panelists on The McLaughlin Group fared no better than a coin flip, in terms of the accuracy of predictions offered on the show. Silver cites to the work of Philip Tetlock, who found that most professional politician scientists, commenting on global events, didn’t fare much better: “....about 15 percent of events that they claimed had no chance of occurring in fact happened, while about 25 percent of events that they said were absolutely sure things in fact failed to occur.” Much of this was because most forecasters were rather ideological “hedgehogs”, type A personalities with strongly defined worldviews, through which they offered predictions. The minority who were “foxes” (less ideological, more nuanced, and open to incorporating work from various disciplines) were actually much more effective in their predictions.

Poker players seem to lose as much money as they win (largely due to overconfidence in their own abilities; as professional player Tom Dwan put it “People can have some pretty deluded views on poker.”), while local weather forecasters offer innacurate predictions more often than they should, as compared to those offered by the National Weather Service (in part due to a tendency by many local news weather reports, to focus more on television ratings, than accurate weather predictions).

So what’s the solution? Silver offers a framework (really, a mental model), through which we might improve our understanding of the world: Bayes’ Theorem.

Thomas Bayes was an English statistician and minister, who became most famous for his eponymous Bayes’ Theorem, which was published after his death. As Silver explains, Bayes’ theorem is mainly concerned with “the probability that a theory or hypothesis is true if some event has happened.”

Bayes’ Theorem is a simple equation, which makes use of three variables: x (our initial estimate of the probability of some event), y (a new event occurs, which is conditional on x being true), and z (a new event occurs, but x is false). Bayes Theorem uses the formula (xy)/(xy + z(1-x)), to calculate probabilities. (Note that while Silver used the variables x, y and z, in his description of the formula, many professional statisticians use a different notation, to refer to the same variables).

Rather than consider it in the abstract, let’s use a highly relevant (and tragic) example from Silver’s work: the 9/11 terrorist attacks, specifically, the two planes that hit the World Trade Center.
Silver first asks us to examine the prior probability of someone flying a plan (as part of a terrorist attack) into the WTC. Assign that probability to the variable x (Silver estimates 0.005%). Next, we consider the occurrence of a new event, which is the first plane hitting a tower of the World Trade Center. Silver considers the probability of this plane hitting the World Trade Center, if terrorists are in fact attacking Manhattan skyscrapers (he assigns this probability, denoted by variable y, a value of 100%). Lastly, Silver asks us to consider the probability of a plane hitting the World Trade Center, if terrorists are actually not attacking Manhattan skyscrapers (i.e. an accident). He assigns this variable a probability z, of 0.008%. Silver then applies the formula for Bayes’ theorem, which returns a result of 38%. This means that when the first tower of the WTC was hit by a plane, there was a 38% chance that the WTC was under attack.

However, we aren’t done just yet. Next, we must revise our probabilities of a terrorist attack, to reflect the fact that a plane has already hit one of the towers of the World Trade Center. Here, x will reflect a revised probability of an attack on the World Trade Center, given that the first plane already hit a tower of the WTC. As stated above, that number is 38%. For y, the probability remains at 100%, while z remains at 0.008%. Applying Bayes’ theorem again, we find a result of 99.99%. This means that if a first plane already hit the World Trade Center, the probability that a second plane hitting the towers, indicates a terror attack, is 99.99%, a “near-certainty.”

As Silver explains, Bayes’ Theorem is so powerful, because it allows us to take account of uncertainty, that is “the limits of our knowledge”, despite our mental biases, and an overflow of information (much of it noise), in the era of Big Data. It forces us to take a stand, and either revise or strengthen this initial position, depending on additional information we receive. He touches upon the findings of medical researcher John Ioannidis, who published an influential paper arguing that the majority of research findings, published in scientific and medical journals, are in fact false. Ioannidis tells Silver that we can now measure “millions and millions of potentially interesting variables” (i.e. more data) but “most are not really contributing much to generating knowledge.” Silver frames Bayes’ Theorem as a means of cutting through the fog, and offering a more rigorous framework to our analysis of various questions, in a variety of fields.


Silver details how a successful sports bettor applies the principles of Bayes’ Theorem to better test and revise his beliefs about a team’s future performance, while computer chess programs (most notably IBM’s Deep Blue), use a Bayesian approach, to explore “the more promising lines of attack.” Google’s culture of product experimentation is grounded in Bayes’ work, as it allows them to quickly experiment with new ideas, with their users, make a hypothesis as to what will work, test predictions, and act on the results (while Google’s self-driving cars make use of Bayesian calculations). Professional poker, like sports betting, offers considerable upside to those who bring a Bayesian lens to anticipating another player’s hand and strategies (revising probabilities as the game progresses, and a player learns more about an opponent’s actual hand). Those who forecast climate change (which has proven to be a political minefield), also benefit from a Bayesian approach.

The thing is, each of us might think more effectively, through real-world application of Bayes’ theorem. In one of his book’s more humorous moments, Silver explains how to use Bayes’ Theorem, to figure out whether your partner is cheating on you, or there’s a different explanation for their unusual behavior.

Computer science professor Allen Downey offers a fascinating example of how he assessed the chances of whether a carbon monoxide alarm going off in his house, was in fact accurate, based on his knowledge of false alarms, factors in his house that might set off an alarm, and other environmental factors. As Downey puts it “Think like a Bayesian. As you get more information, update your probabilities, and change your decisions accordingly.”  

Interviewed by Gizmodo, mathematician Spencer Greenberg explains how someone might gain a sense of whether a new workout regimen has in fact increase how much extra energy they have have, by looking at prior energy levels, how these have changed, and any other plausible explanations. As Greenberg sees it, Bayesian thinking in everyday life, isn’t really about calculating Bayes’ formula, so much as it is a means of testing and updating our beliefs, by considering the strength of various pieces of evidence, correcting “glitches” in our thinking, and avoiding “absolute certainty”, which often leads us astray.

We live in times of rapid change, and considerable uncertainty. The trajectory of the global economy remains unclear, while political/policy matters, across the globe, remain mired in doubt. Artificial intelligence is changing the world at a brisk pace, and will transform both the nature and availability of work. The lifespan of the largest and most profitable companies is growing shorter and shorter. Revolutionary technologies like CRISPR are transforming genetic engineering, at a breakneck pace. In 2016 and 2015 alone, more data was created, than in the previous 5000 years of humanity.

In this climate, it’s more important than ever, that we have some useful frameworks, to better understand the world around us. Bayes’ Theorem offers an effective way forward, one that we can all make use, in ways large and small, in our personal lives.





















































Wednesday, May 31, 2017

Regulating The CRISPR Revolution

Credit: Sciencenews,org

The government of New Zealand recently announced an ambitious plan, to eliminate rats, possums and stoats from the nation’s ecosystem, in order to protect wildlife (through both eliminating competition for resources, as well as disease spread by these animals). As Singularity Hub explained, this initiative is possible, thanks to what’s known as a “gene drive.”
A gene drive takes advantage of the fact that most animals have two copies of each same gene, and sometimes, these copies are different variants. On occasion, a variant might have been edited to promote a desired trait, let’s say, infertility (so that pests can’t reproduce). Normally, any offspring of a parent who held this gene, would end up containing both the infertility gene, as well as the normal gene.
A gene drive seeks to remove the natural version of the gene (using genetic engineering techniques), and replace it exclusively with the edited gene, which will then be passed on to future generations. If an animal with the edited gene breeds with a normal animal, the gene drive seeks to again replace any natural genes with only the edited variant. In a matter of around 10 generations, 100% of a species would carry this gene (in this instance, with an inability to reproduce, the animal would become extinct).
If all of this sounds a bit fanciful, keep in mind that versions of this approach have already been successfully implemented . Dr. Anthony James of the University of California at Irvine, and Dr. Valentino Gantz, at the University of California, San Diego, were able to spread a gene for resistance to malaria, through generations of mosquitoes, at a 99% efficacy rate. This gene isn’t really useful to mosquitoes themselves, and so under the laws of evolution, it should have ceased being passed along after a certain number of generations. Yet, thanks to this gene drive, the opposite happened.
American scientists aren’t the only ones who’ve been working on mosquito gene alteration. Dr. Austin Burt at Imperial College London, and his team of collaborators, carried out a succesful gene drive to target female reproduction in mosquitoes, such that future generations of mosquitoes might eventually become extinct.
James and Gantz’s team, as well as Burt’s, were able to implement this gene drive thanks to CRISPR, and more specifically Cas9, a gene-editing, technology which makes it easier to splice and make changes to DNA. Cas9 (which stands for CRISPR associated protein 9), plays a major role in why CRISPR offers such a revolutionary method for sustainable gene editing.
Because Cas9 can snip any DNA sequence, it can be added to mosquitoes in the embryo stage, to insert a malarial antibody (which helps to ensure a resistance to malaria, in James’ and Gantz’s experiment), or a gene to hinder reproduction (in Burt’s study). This Cas9-centered approach was made possible in part thanks to a 2015 experiment by Gantz, and his UC San Diego colleague, Dr. Ethan Bier. They used Cas9 to conduct a gene drive to spread albinism among fruit flies.
There’s little question that these findings are revolutionary, and can have a transformative impact on our planet. After all, who wouldn’t want to stop the spread of malaria, or prevent the growth of diseases, which threaten to harm the remarkable wildlife found in New Zealand?
The problem is, it isn’t quite that simple. Elimination of mosquitoes (or at least engineering their immunity to disease) might have a positive overall impact, especially given their role in spreading diseases like malaria and Zika to humans. Yet, these insects are also a source of food and flower pollination, and we simply don’t understand enough about their role in our ecosystem, to accurately predict the long-term consequences of their elimination. How might New Zealand’s efforts to curb certain species, impact it’s environment balance? While rats and possums aren’t keystone species, that is, absolutely critical to our environment, each plays a role of note in our world, and the eventual impact of their absence, remains uncertain.
Let’s also keep in mind that scientific experimentation and advancement, is rarely limited to just one specific area. If genetic engineering of wildlife poses concerns, what about attempting similar techniques on human beings? In April 2015, researchers at Yat-sen University in China, led by Dr. Junjiu Huang, published a paper, announcing that CRISPR had been used to modify some non-viable human embryos. The Yat-sen team removed a gene sequence which carried the blood disorder Beta thalassemias, and inserted a healthy sequence in it’s place. The study at Yat-sen University ultimately was of limited effectiveness (only a small fraction of the embryos initially tested, ended up with the healthy sequence), although one rather concerning byproduct of this experiment, was that CRISPR also “unintentionally caused mutations in other parts of the genome.” In other words, CRISPR carried some unpredictable results, making genetic changes that weren’t expected.
Huang’s research created quite a stir within the scientific community. The prominent science journal Nature, refused to publish these results, in part because of ethical concerns, while a number of well-known geneticists wrote a piece in the journal Science, counseling scientists to avoid running CRISPR techniques on human embryos.
Why was Huang’s experiment so controversial? Lots of the researchers who criticized Huang’s work in Nature, were advocates of and experts in human genome modification, primarily for the purpose of preventing disease. However, their personal research, and that which they endorsed, focused primarily on editing of somatic cells (i.e. T cells).
Geneticists have made tremendous progress in their applications of CRISPR with T cells. It has been used to remove HIV genes “from the genomes of mice and rats infected with the virus”, and to restore sight to blind rats. In November 2016, a team led by Dr. Lu You, an oncologist at Sichuan University in China, injected a cancer patient with cells that contained CRISPR-edited genes, in an attempt to both test the safety of this use of CRISPR, and to explore how these treatments might actually work. This effort is being followed by the world’s first CRISPR clinical trial, where researchers, led by Dr. Edward Stadtmauer, will remove T cells from cancer patients, and perform several CRISPR edits, testing both safety and effectiveness.
Embryonic editing might offer excellent opportunities for scientific advancement, but also, increased chances of unpredictable changes or cuts, being made elsewhere in the genome (as happened with some of the embryos in Huang’s work) These modifications might not be known until after birth of the embryo, and could well be passed on to an embryo’s future children, creating additional risks for generations to come.
Observers have also raised the prospect of “designer babies”, basically, future human children, who have been engineered to either lack or carry certain traits. There are a number of ongoing efforts around CRISPR-based germ-line engineering (editing of egg and sperm DNA, or of embryos themselves), being conducted by scientists at Harvard, MIT, and several universities and companies abroad.
According to Luhan Yang, a postdoctoral researcher at Harvard, these efforts might make it possible to one day eliminate familial vulnerability to genetic diseases like cystic fibrosis, or protect future generations against infection, Alzheimer’s, or even simple aging. Other researchers hope to use CRISPR-based embryonic cell modifications to reverse autism.
Yet, there are also some morally troubling aspects, to this sort of research. What if research advances far enough that some (wealthier) families use these tools to engineer specific traits, such as a certain eye color, or physical strength or agility, or intelligence (to the extent that physique and intelligence are genetic traits)? What does it mean for social and economic inequality, when families of means, can provide their children with yet another powerful advantage, from the day they were born? How about the unforeseen consequences of genetic modifications, as has occurred with both plants and animals? How much more impactful would such missteps be in humans, who have, as a whole, been “optimized” through millions of years of evolution.
As a solution, we ought to implement some basic parameters, applicable on an international basis (let’s remember that Dr. Huang’s research is being conducted in China, not the United States) , to regulate genetic modification and engineering, across species (both humans and animals), and types of research (i.e. gene drives in insects, or T cell and embryonic cell modification in humans). At their core, these rules must balance the drive for innovation and progress, with an eye to the very real risks posed by human interference, in the code of life.
What might this regulatory structure look like? First, we would need to bring together nations across the globe, to ratify a treaty, whereby they agree to implement the recommendations of a panel of experts, assigned to develop guidelines around these questions.
This panel will be made up of a broad cross-section of expert scientists, both geneticists and physicians, who focus on somatic, as well as germline (including embryonic) editing/modification. To this group we will add prominent bioethicists, who are tasked with grappling with complex ethical issues, in both biology and medicine. While expertise in one’s respective field, ought to be the most important requirement for membership on this panel, it also makes sense to select individuals from a fairly broad range of nations, so that their recommendations are not questioned as the biased, unfair assessments, of scientists from a few select nations.
These individuals will need to periodically (perhaps every 18 months or so), offer a consensus view, around what sort of experimental research ought to be permitted (and restricted), as well as any notable risks on the horizon, particularly in terms of germline modification practices. Given the difficulties that people seem to have in reaching unanimous agreement on even simple matters, the group should be allowed to make recommendations based on substantial, although not necessarily unanimous agreement. 
For this reason, any recommendations issued by this panel, ought to carry the support of at least 2/3 of all members, with dissenting members offering their own thoughts on how to proceed (sort of like those justices who end up in the minority in a US Supreme Court ruling).
Next, those nations who were signatories to the treaty (and ideally, the treaty can be expanded to include as many nations as possible), will ensure compliance by scientists within their borders. In a sense, this might not be so hard, because scientists typically publish the results of their work, for peer review, making it possible to catch violations. Yet, we can also imagine rogue scientists, quietly abetted by nations which choose to look the other way, given the potential for profit. That must be prevented.
It’s not hard to see the challenges (and the potential flaws) in this entire process. The international community has faced serious challenges in reaching actionable agreements around issues like climate change, and the proliferation of nuclear weapons. Genetic modification isn’t likely to be any less contentious (or easy to enforce), particularly when a country whose scientists are unhindered by these rules, might stand to gain billions of dollars, by successfully commercializing certain technologies. Also, given that humans aren’t very good at predicting the future, what’s to say that scientists don’t end up completely overlooking potential risks of genetic modification?
When it comes to biological innovation, we are living in an incredibly interesting time. CRISPR/Cas9 are starting to make it possible to alter the code of human life, in a way that can fundamentally change human and animal existence. Some of these changes might be very positive. Yet, such power also carries the potential for massive missteps, with potentially catastrophic consequences. Let’s proceed with caution.

Monday, March 13, 2017

Make The US Citizenship Test A High School Graduation Requirement

                                                    Photo Credit; Whitehouse.gov

Feeling down about the future of this country? Searching for some inspiration?
Go watch a group of new Americans take their oaths of citizenship. Typically conducted before a federal judge, often in a large convention center, you’ll see hundreds, sometimes thousands of people, from every corner of the globe, vow to “support and defend the Constitution and laws of the United States of America against all enemies, foreign and domestic”, and to “bear true faith and allegiance to the same.”
In this room, you find men and women who are brimming with pride and optimism, often having waited a decade (or longer), for this honor. Each of these individuals is here for a unique, deeply meaningful reason.
Some fled horrific persecution and violence. Others believed that the economic and educational opportunities they would find here, far surpassed anything their homeland had to offer. For many folks, America offered an opportunity to reunite with their families, and build a new life together.
You won’t fail to leave the room more hopeful about the values which our nation, at it’s best, stands for, and with a deeper understanding, of what it really means to be an American. Let’s keep in mind, however, that for immigrants, the privileges of citizenship didn’t simply appear out of thin air.
If you weren’t born in this country, and didn’t qualify for citizenship through your parents, you are required to pass an exam, which tests your knowledge of American history and government. Out of 100 potential exam questions, prospective citizens will be presented with 10 questions, and required to correctly answer 6 of them. This test is conducted only in English.
The citizenship test covers a range of topics, but is primarily focused on the Constitution, the structure of American government, the basics of US history and geography, and the rights and responsibility of citizens. Questions range from which are rights guaranteed by the First Amendment, to how many members the US Senate has, the location of the Statue of Liberty, the reasons the Civil War was fought, and more. In short, this exam covers the basics of American history, government, and rights, as derived from the Constitution.
While aspiring citizens are put through this process, many Americans who enjoy citizenship through birth or parentage, lacked even basic knowledge, of many of the topics covered in this exam. In 2011, Newsweek arranged for 1000 Americans to take the citizenship test, and found that 29% of those tested were unable to name the vice president, while 44% couldn’t define the Bill of Rights, and 73% were unable to explain why the United States engaged in the Cold War.
In 2012, another study, conducted by the Center for the Study of the American Dream at Xavier University, found that 1 in 3 Americans were unable to pass this exam, which required answering 6 in 10 questions correctly (meanwhile, 97% of immigrants who took the test passed) . While examinees fared better on questions of history and geography, they didn’t do so well when the topic was the Constitution, or identifying current elected officials. 75% of those surveyed did not know the function of the judicial branch of our government, while 71% were unable to identify the Constitution as the supreme law of the United States, and 63% could not name at least one of their state’s members of the US Senate.
The point isn’t to shame those who lack basic knowledge of history and civics. After all, while it would be nice if everyone displayed a certain level of intellectual curiosity, is it truly reasonable to expect folks to know much about something which might not have been covered in any real depth, during their K-12 education? Given that education in this nation is primarily a state and local matter, and many school districts don’t prioritize these topics, it isn’t hard to see why so many people are lacking knowledge.
However, it is more crucial than ever, that we cultivate such learning. Let’s just take a look at everything that has happened, over the past 15 months or so.
We’ve watched a heated debate unfold, regarding the balance of power between the judicial and executive branches of government, particularly on immigration matters, after President Trump issued an executive order to restrict entry into the United States, by nationals from 7 Muslim-majority nations. The administration’s proposed immigration policies have also raised questions about the role of cities and states in enforcing federal laws, as some localities have vowed to resist federal efforts to deport more undocumented immigrants.
Meanwhile, impassioned, sometimes violent demonstrations against former Breitbart provocateur Milo Yiannopoulos, led to the cancellation of his scheduled appearances, on a number of college campuses. This has prompted much discussion around the meaning and application of the First Amendment freedom of expression. Similar conversations have focused on freedom of the press as well.
In this fractious social and political climate, which doesn’t seem likely to grow any less contentious over time, it is more important than ever, that each and every American understand the basic rights and responsibilities of citizenship, as set forward in our Constitution. It is also crucial to have some idea of the structure of our government (especially in terms of how power is allocated between state and federal government, as well as the different branches of government). Of course, it certainly make sense to know which elected officials represent us in Congress (after all, how else can we petition them?), and lastly, to know something about American history (we most effectively shape our future, when we take into account what happened in our past).
Towards this end, high schools ought to implement a civics requirement. In order to receive a high school diploma, one would be required to take a class that covers the topics discussed earlier, at a basic level, and pass an exam (similar to the US citizenship exam), which tests their proficiency in these topics.
I’m not under the illusion that each student who completes this course and exam, will be some sort of historical or constitutional scholar, waiting to take his or her seat on the faculty at Princeton or Stanford. If the math and science test scores of American 15-year-olds are any indication, we’ve got much work to do, when it comes to education in general. Creating history and civics literacy won’t be simple. Also, the possibility of teachers “teaching to the test,” that is, focusing on helping students pass exams, over cultivating deep knowledge, is a real risk, and one we’ll have to find a way to work around.
Yet, if we are honest with ourselves, we don’t have a choice. We have to find a way to push Americans to learn these topics. Centuries ago, Thomas Jefferson observed “If a nation expects to be ignorant and free, in a state of civilization, it expects what never was, and never will be.” This statement remains true today. If we wish to remain a democratic republic, Americans must better understand the key concepts which underpin our Constitution, government, and history, and, at some level, reflect on what it will take to live up to the very best of our heritage, in the years to come.