Lambda School Twitter fight, ISAs, Incentive Alignment, and Outcomes Reporting
I started writing this section for yesterday’s newsletter, but as news kept coming out and I kept adding more thoughts, it got too long, so I pulled it into a post of its own.
There’s been a lot of Twitter back-and-forth, and that medium facilitates picking a side and aiming for dunks - playing to the folks who already agree with you. This post is my attempt to understand what’s going on, without trying to judge a winner or rebut anyone. I explicitly focus on areas of agreement between the two ‘sides’ that I map out.
I’ll summarize the recent Lambda School news and the various tweets, subtweets, and threads, then attempt to go full galaxy brain and try to understand why the fight is happening.
Note that I might be wrong, there’s a lot at stake here for students, neutrality isn’t the ‘correct’ mode or opinion, and that both critics and proponents have a ton of valid things to say. Acknowledging my own biases - I’ve been involved in bootcamps (but not Lambda in particular), and have a general pro-bootcamp sentiment.
Context and Terms
In case you’re just catching up on all this, Lambda School is a popular learn-to-code bootcamp that markets itself to career changing adults. It’s received a lot of favorable media coverage, raised close to $50 million in funding, and attracted lots of online students.
Part of its rise is due to the popularization of a new kind of student financing agreement called an Income Share Agreement, or ISA, where students pay for their tuition as a percentage of their income after they get a job after the program. Particularly in light of mounting student loan debt in the United States, ISAs have been touted as a more fair way for students to pay for their education. Lambda School’s founder Austen Allred has touted ISAs both for their student protections and for the incentive alignment that ISAs ought to create between the student’s interests (nominally, getting a high-paying job) and those of the school (making more money).
Lambda in the news
There were three big Lambda school articles, two in the Verge and one in NY Mag, and a few scattered reaction pieces.
The Verge kicked off with an article The High Cost of a Free Coding Bootcamp, which reported on students who were frustrated by different aspects of Lambda’s program, with particular focus on a UX cohort that had gone poorly.
The next day, they followed up with As Lambda students speak out, the school’s debt-swapping partnership disappears from the internet, detailing how Lambda had raised money from Edly backed by its ISAs - a financing scheme that the Verge claimed diluted the promise of the school’s advertised incentive alignment.
NY Mag’s Intelligencer followed with Lambda School’s Misleading Promises, which found an internal communication (an investment memo’s section on risks) with a lower figure for Lambda’s job placement than the 86% advertised.
Lambda’s CEO Austin Allred published details about the new ISA financing the same day as the NY Mag piece, (a deal strikingly similar to Kenzie Academy’s debt raise last November).
The Internet reaction
There were scattered reactions on Twitter and across the web. Several big tech personalities got involved.
I am not going to link to the Tweets, but there are plenty of high-profile people on either side. I trust that you can find satisfying dunks and hot takes. There’s plenty of smart and compassionate commentary on both sides, along with the snark.
Two interesting pieces to call out.
This Twitter thread from Louis Gelinas (current Lambda student):
The thread is long, but worth reading. It’s notable because it recognizes complexity, and speaks to both the experience on the ground as well as the facts - and how they can be presented. It has a generally pro-Lambda position, but got positive responses from folks who have taken publicly skeptical or critical positions.
The other take that I think is worth sharing was this weeks newsletter from Ranjan Roy in The Margins (another Substack newsletter). Ranjan comes from the world of finance, and goes into depth about his change of heart from thinking of financing ISAs as unquestionably good and needed, to having more doubts.
Possibly the biggest lesson I took away was, the more distant a risk becomes, the more distorted all the related incentives become.
My thinking has followed a similar path. My initial reaction to the Verge piece on the financialization of ISAs was that it misrepresented the situation and ignored the reality of having to actually pay teacher salaries to run the bootcamp, and make the cash-in-the-bank work while students trickle through to the tuition-repayment phase of their journey.
After reading the Margins piece, I start to see why selling pieces of ISAs might actually change the on-the-ground incentives for Lambda. It means they can be (and are incentivized to be) more aggressive about recruiting students, since they can get some of the money up front. Otherwise, they’d have to focus on getting the current students graduated and paying before they could grow more.
Ultimately, that may be a better set of incentives. I think bootcamps should be bigger, and marketing and recruiting is an important way that education pathways other than 4-year degrees will become popular and accepted.
Still, the set of incentives created by raising debt based on ISAs makes a material difference to what their strategy will be, and it’s different from what Lambda’s advertised.
What’s all the fighting about?
I keep returning to the question - why is Lambda School in particular getting all this scrutiny?
They aren’t the biggest bootcamp — not by a long shot — I don’t think they crack the top five! They also aren’t the only new online bootcamp to raise a bunch of money - Kenzie Academy raised $100 million in debt in November. They aren’t the only bootcamp to feature ISAs - in fact, nearly all of the popular bootcamps offer an ISA-like financing option.
I think the answer here takes two steps. First, Lambda gets a lot of media attention because its founder is excellent at using Twitter for promotion. That triggers a name-recognition among tech media folks, who then mention Lambda in coverage of student debt and the rise of ISAs, and Lambda manages to get several favorable profiles in tech media. That draws more eyes to Lambda, which has an ‘everyone-is-talking-about-it’ feedback loop, drawing in more eyes.
Second, of a piece with broad tech backlash, more critically-inclined people in the media are on the lookout for mistakes, failures, and complaints about Lambda. When criticism does emerge, it gets picked up by folks who are predisposed against the model, for their idiosyncratic reasons. Because Twitter facilitates fighting, dunking on people, and subtweets, we get a lot of that.
The reactions seemed to lump broadly around two poles. One is the pro-Lambda cluster, and seems to broadly track with venture capital, Y Combinator, pro-bootcamp, and pro-ISA accounts — a “building things is hard and takes courage and financing” camp. The other pole is characterized by skepticism of big tech (venture-capital-backed tech in particular) and is more critical of bootcamps, ISAs, and Lambda.
As I read the articles, tweets, and subtweets, I feel my body tense and my heart beat faster. When I see the back and forth on Twitter, the tone I hear in my head is angry, frustrated, or smug.
I’m going to take a piece from Anatol Rapoport’s rules for criticizing with kindness, as popularized by Daniel Dennet’s Intuition Pumps and Other Tools for Thinking - particularly, Rule 2:
List any points of agreement (especially if they are not matters of general or widespread agreement).
So, what beliefs are shared between all sides?
ISAs (with appropriate regulation) can protect students from predators and align incentives between students and for-profit schools
Bootcamps help lots of people break into the tech industry, taken broadly, and as many jobs in the industry are currently high paying, this can have a transformational impact on those students financial fortunes
Pathways and access to spaces and industries that are traditionally hard to access is a positive good
Some people don't succeed in bootcamps, whether they don’t get in, don’t graduate, or don’t find a job afterwards
It is a personal hardship when someone starts a bootcamp and ends up dropping out, or graduates from a bootcamp and doesn’t find a job
Prospective students should have accurate information about their chances of reaching their goals when deciding on an educational product
Usually, for bootcamps, this means the chance of graduating and finding a job
Good regulation can help protect students from scams and "certificate mills"
There are, and have been through history, scams and certificate mills that prey on students, particularly on students from vulnerable populations, military veterans as a prominent example
Some Lambda students have found success in the program, and their outcomes have been transformational
Some Lambda students have found the program deeply lacking, and have asked to be released from their enrollment agreements, and some have actually been released
Finding the disagreement
As far as I can tell, there’s room for nuance and complexity in this narrative - and there’s lots of places where people agree.
But, with all the fighting, there must be some critical claims at the heart, some actual points of contention.
Here’s my summary of the critical claims about Lambda School, and my understanding of how deep the contention runs.
Lambda hasn’t gotten the necessary approval from the California Regulator, which issued a scary-sounding action against them
My dive into the history of California bootcamp regulation found that this probably isn’t something that’s super concerning. Lambda’s public statements are pro-regulation, they’ve said that they’re complying with the regulator, and from the looks of the other bootcamps that have had similar action taken against them, it usually disappears after the bootcamps pay a fine.
This fits an existing media narrative about tech startups being willing to break the rules to get what they want, but it looks like this might just be par for the course for bootcamps in California. I don’t actually think that students suffer from this, nor do I expect the regulator to meaningfully impact the student experience.
Lambda students’ learning experience is poor
The core claims here seem to be that the curriculum is underdeveloped and that the teachers aren’t experienced - particularly the coaches, who Lambda hires from the student body, and who have the most interaction with students.
Much of Lambda’s curriculum is open source, so you can see it. Curriculum is only one piece of a learning experience, but from a cursory inspection, it doesn’t seem obviously bad. As a curriculum designer myself, I am sure there are lots of places it could improve - Lambda is probably aware of lots of those places, and is probably working on improving them.
Lots of schools, including most bootcamps, hire TAs and coaches from their student bodies. Having worked with a ton of amazing coaches drawn from the population of bootcamp grads, I think this is a good model. It promotes social learning and gives students a concrete picture for where they will be soon. Coaches are close to the student experience, and so can effectively cross the knowledge and understanding gap that arises between experts. Maybe Lambda should pay its coaches more. From student reports about expertise and coaching style, it sounds like there are some coaches who should get better training (or who shouldn’t be coaches), but that doesn’t invalidate the coaching model.
Financing ISAs meaningfully changes the school’s incentives
Financing ISAs means the school gets some of the money in advance, in exchange for some of the ISA payments in the future. In the bootcamp business model I shared yesterday, I highlighted why any new bootcamp needs something to fill this gap.
Schools need cash to pay for running the school long before graduates start paying back their ISAs. Whether this comes from a loan, from raising seed money from VCs, having students pay tuition up front, or financing ISAs, every school needs to solve this bootstrapping problem.
As noted above, financing ISAs does change the strategy that the school will follow. It can focus on growing faster, and it can last longer before a failure to place students would shut the school down.
Still, the school will make more money when students get paid more, and they will eventually fail as a business if the students don’t do well enough. I can’t say whether this invalidates the school’s claim that ‘we only make money when you do’. It seems to me that they do have cash in the bank before students get placed in their jobs, but also, the financial outcome of the business is still inextricably tied to that of the students.
Marketed placement rates don't match actual placement rates
Similar to the claim about financing ISAs, there is a false-advertising core to this claim. Prospective students should have a clear picture of their chances of graduating, and upon graduation, of getting a job - especially if that’s the core of the marketing message.
The job placement rate is central to the heart of lots of bootcamp marketing - Flatiron School, where I worked, has the Annual Outcomes Report as a centerpiece of its marketing message.
Unfortunately for everyone, things are hard to count. Should a bootcamp be on the hook, outcomes-reporting-wise, for students who decide after the first week that the program isn’t for them? That might be a failure of the program, or it might just not have been interesting for that student. Lots of students who enroll do not finish the program. How should those students be counted?
Bootcamps also tend to exclude from their outcomes reporting students who start with structural barriers to employment where the bootcamp is located. In the US, that often means that folks who would need a work visa to get a job don’t get included as ‘eligible ‘ or ‘job-seeking’ students in bootcamp outcomes reports. Similarly, a lot of students stop responding to emails after they graduate - it’s hard to tell if they got a job, are still looking, or have given up. In the detailed reports that bootcamps publish, these details show up, but they get left out of the top-line number.
My post yesterday termed the product of the graduation and job placement rate the ‘prospective success rate’. In effect, that’s the expectation that a student should have, on average, of completing the program and getting a job - an all-in number for your odds of success. This seems like it would be the fairer number for students to have in mind when they’re thinking about whether they should attend some program.
Lambda’s leadership’s response to criticism
Critics want Lambda’s leaders to hear the criticism, believe it, and take action to correct the deficiencies in their program. Often, students report feeling that their voices were unheard or brushed off.
These criticisms land for me. I think listening and responding to criticism is good, and that Lambda can probably do better at this.
Who are we comparing Lambda to?
If we’re comparing Lambda to other bootcamps or to traditional higher ed, how does it stack up? How does it compare to self-study, or taking free online courses? Or, are we comparing each decision the school has made against an idealized version of the school, where the decision was handled ‘better’?
Choosing the appropriate comparison group is challenging. It frames the rest of the discussion, but it’s also hard to explicitly mention that frame. When someone chooses a comparison group, someone else might choose another.
Compared to colleges
Lambda seems to be pretty different. It’s all online, it’s shorter, students pay later and are protected from the student loan debt of traditional colleges. It’s not accredited, it’s not as widely known or understood, and it doesn’t behave like college.
Colleges also don’t report, generally, on their job placement numbers. Their graduation numbers vary, but don’t seem to stack up particularly well against Lambda. The six-year graduation rate of students first starting a 4-year bachelors degree is 60%. Some schools - particularly elite colleges and universities - do much better. Other schools - particularly for-profit colleges - do much worse. The US national average for for-profit schools 6-year graduation rate is a 21%.
Compared to bootcamps
Lambda seems broadly similar to other bootcamps. It has a longer program than lots of bootcamps, but not the longest. It’s all-online, but there are other all-online bootcamps. The graduation rate and job placement rate are neither the best or the worst among bootcamps.
It’s got a lot of media attention, and it’s what people on Twitter are paying attention to.
Outcomes and Transparency
Everyone cares about the actual student outcomes numbers.
Bootcamps, especially because they have used their job placement rates as a cornerstone of their marketing. Perhaps rightly, they’ve had lots of criticism about the way they’ve reported (or not reported) those numbers.
For Lambda, there’s a 15-month lag between when a student starts the program and when they are 6 months past graduation, the cutoff for inclusion as a successfully placed graduate. Since compiling an outcomes report that you can trust takes time (some bootcamps use an external auditor, a necessarily slow and careful process), bootcamps usually report on outcomes once or twice a year. That means they are usually reporting on students who started the program two years before!
This structure leads to a lot of calls for more transparency from Lambda School, with simultaneous claims from defenders that Lambda is better than most schools in terms of outcomes reporting and transparency.
On Monday, Austen shared the current numbers breakdown in a twitter thread, in advance of a more detailed (and audited) version:
So,
Reporting is hard, many caveats from above
72% Graduation Rate
78% Placement Rate
It doesn’t seem like advertising an 85% placement rate was deceptive, based on their previous outcomes.
From the list of points of agreement above,
Prospective students should have accurate information about their chances of reaching their goals when deciding on an educational product
I’m sure that when Lambda has audited their new outcomes report, they’ll update their marketing to reflect it.
What would a ‘good’ placement number be?
Some bootcamps have a high admissions bar, and manage to graduate and place most of their students. They sacrifice access, and sometimes explicitly market their exclusivity. Other bootcamps let all students start, and end up with many students who drop out or don’t get a job.
I don’t foresee any bootcamps publishing a ‘prospective success’ figure that includes acceptance rate, graduation rate, and job placement rate. Maybe someone like Career Karma or Course Report sits in the kind of position to build that kind of tool.
If a school could drive up all three numbers simultaneously, it would be a Pareto improvement over all the other schools. I don’t doubt that there are real differences in quality between schools, but I’m skeptical that focusing on the job placement number alone helps understand what’s going on - or incentives the right behaviors from bootcamps.
What’s missing from the Lambda / ISA discourse?
All schools have some skin in the game
Reputational risk matters to schools that aren’t bootcamps. Every school’s ability to attract students and faculty and place their alumni in jobs depends on the impression they create.
Incentive Alignment isn’t sufficient
There is no invisible hand that teaches the students to code.
Schools with poor incentive alignment are many, and they are often predatory - see the previous post’s aside on cosmetology schools.
But for all the talk of Incentive Alignment, it all has to hit bedrock with differences in students’ experience. For the incentive alignment to mean anything, you have to actually innovate in teaching, in placements, in community, in curriculum, or in something else that is actually different for the students. The financial experience of students is real, and different under ISAs than under traditional loans - but it is only a part.
We don't really get to see into the inner workings of most schools, and few critics, students, or investors can get enough information to tell whether the students will actually learn differently.
So, what’s the galaxy-brain take on Lambda School?
I have been trying to figure out where I stand on the debate. I have been trying not sign up for one ‘team’ or another, since my identity will get wrapped up in that side, and that might blind me to what’s actually going on.
I keep coming back to this quote about AI alignment believers and skeptics (with a slightly twisty origin):
The “skeptic” position seems to be that, although we should probably get a couple of bright people to start working on preliminary aspects of the problem, we shouldn’t panic or start trying to ban AI research. The “believers”, meanwhile, insist that although we shouldn’t panic or start trying to ban AI research, we should probably get a couple of bright people to start working on preliminary aspects of the problem.
We can probably formulate something similar here.
The skeptic position is that, although bootcamps and ISAs are important models to explore, we need to protect students from poor educational experiences and deceptive marketing - after all, education is supposed to be a pathway to a better life. Proponents’ position, on the other hand, is that education is supposed to be a pathway to a better life, and, although we need to protect students from poor educational experiences and deceptive marketing, Bootcamps and ISAs are important models to explore.
This is more than I wanted to write about Lambda. They get a lot of attention already, a lot of what I have to say has already been said well already, and following the controversy closely just invites stress into my life.
Still, I think this helped me see some of the threads of the argument on either side more clearly, and hopefully it did the same for you!
If you have thoughts or feedback (or if I got it all wrong), please let me know - you can just reply to the email, and I’ll get it!
Thanks for reading!