A short history of bootcamp regulation, the state of the bootcamp market, and learning roadmaps
This week, we revisit the ongoing Holberton (and Lambda School, and tradecraft) regulation story with a look at bootcamp regulatory action over time. Also: highlights from the Career Karma Bootcamp Market Report and some thoughts on CS learning journey roadmaps, with links to resources.
Since this is only the second issue, I’m particularly interested in your feedback. Responses should make it to my inbox, and comments are on for the permanent version on Substack.
Let me know what you think!
A Short History of Bootcamp Regulatory Action in California
In last week’s newsletter, I linked to an article about the California Bureau for Private and Postsecondary Education action against Holberton School.
For those keeping score at home, this comes in the wake of lots of news about a similar action against the popular Lambda School and as-far-as-I-can-tell-unreported action against tradecraft, a product design bootcamp.
While looking up notes on BPPE’s enforcement actions last week, I saw that they list all of their disciplinary actions on their website. That prompted me to take a deeper look at the different regulatory actions that California has taken against bootcamps over the years.
BPPE lists about 700 schools that it has taken some action against. I ran the list of names against the a list of bootcamps from Course Report, and quickly checked others whose names looked like bootcamps.
Note - I could have missed some bootcamps or regulator actions - it’s a little hard to tell with some of the names. For example, Computer Institution of Technology looks like tech vocational training, but it doesn’t seem like a bootcamp in same sense of the other bootcamps listed here. If you see something I missed, let me know!
Here’s the actions I found, as an Airtable view.
(click through to view the full table)
Bootcamp Regulation Scorecard
To understand how the Holberton, Lambda, and tradecraft stories are likely to play out, we can look at how BPPE actions have resolved in the past, and their timelines.
So, what’s happened for other schools facing regulatory action?
Closed after regulatory action: 3
Still operating, after successful resolution of regulatory action: 10
Closed for other reasons (after a successful resolution): 1
Unclear: 1
Still operating, with pending regulatory action: 3
Closed after regulatory action
Coding House
Origin Code Academy
Orange County Code School
From the Course Report reviews and media coverage, the Coding House closure seems like a clear tally in the regulator’s column, though the school might have shut down anyway. It was bad.
The founder of Origin Code Academy blamed the regulatory action for the closure, and it’s not so clear that the school was bad. The Origin Code Academy Course Report reviews were mostly positive, for whatever that’s worth.
From what I can see, it looks like Orange County Code School is closed. Their website is down, and their twitter account has not tweeted since September 2018, but I’m not 100%. There’s no media coverage of closure or announcement from the school itself (that I can find). Given that it’s not clear that they closed, it’s even harder to tell if the closure was due to the regulatory action. ¯\_(ツ)_/¯
Still operating after successful resolution of regulatory action
App Academy
Code District
Codesmith
Coding Dojo
GrowthX Academy
Hackbright Academy
Horizons School of Technology
Learn Academy
Product School
Rithm School
Closed for other reasons (after a successful resolution to the regulatory action)
Rocket Space’s Rocket U
Rocket Space appeared to have complied with the regulator, then shut down in favor of leaning into its coworking space model. They are still alive as a coworking space, but it doesn’t look like they teach anymore. Unclear whether the regulatory action had anything to do with it, since they did end up with approval.
Unclear
Codify Academy
I can’t tell if Codify’s situation is pending or resolved. May 1 of 2019 shows both a “Withdrawal of Citation” and a new “Citation Assessment of Fine and Order of Abatement”. The Codify Academy website shows they are still registering and teaching students for their part-time online software engineering program. If you know more, give a shout!
Still operating, with a pending regulatory action
Holberton School
Lambda School
tradecraft
I’m on the edge of my seat! Holberton was supposed to shut down this week, but definitely looks like it’s still open…
…but I may have to wait.
From the past timelines, it looks like it’s usually 6 months to a year for bootcamps from the first citation to resolution. Now, Holberton is the only school in the data with an ‘Emergency Decision’, so it could be that BPPE and the California Attorney General will move faster on this one.
It’s not totally clear what the normal timeline is for a new school to become approved through the ‘normal’ process (i.e. not getting cited at all) but looks like it may be similar - about 6 months or a year.
Answered and Unanswered Questions
Q: Are the schools that haven’t been listed here approved? Is there a list of all the approved schools?
A: No, and Yes. Well, sort of. BPPE published the 2017 summary of all the schools that have been approved, and has a search page for currently approved schools. I haven’t got a list of all the bootcamps that are currently operating in California, and I haven’t tried to check those schools using the search page. Some schools are approved, and some are neither approved nor have they had action taken against them. We can’t see if they have an application pending, either.
Interestingly, Holberton still shows up under the list of approved schools. This isn’t so surprising - the emergency action describes that the school was approved, conditional on not offering ISAs. When Holberton kept offering ISAs, they were in violation of the agreement with the regulator. The approval itself, with that condition, isn’t publicly listed anywhere, as far as I can tell.
Q: Per BPPE, schools that have a physical presence need to apply for approval. Do online providers with students in California also need BPPE approval?
The California Education Code provides that all private postsecondary educational institutions with a physical presence in the state of California must apply for approval to operate with the Bureau for Private Postsecondary Education (Bureau). (BPPE)
A: Yes, as of June 2017. This is documented on BPPE’s Out-of-state Institution Regulation page. There’s also a complaint process for CA students of out-of-state online schools.
The new law applies to private institutions that do not have a physical presence in California but enroll California students in distance education, except for non-profit accredited institutions.
Q: For the schools whose last action was a citation or update, but still look open - are they operating illegally? That looks like it usually ends with either the closure of the school, or a fine and closure of the case. What’s the status, in the eyes of the regulator, of the following schools:
Codify Academy
Orange County Code School
tradecraft
Lambda School
Holberton School
This is still unanswered for me.
A related question has to do with the closure mandate in citations. The text of many citations says that schools must shut down and file a closure plan before they can be approved and reopen. However, schools that end up resolving their cases don’t appear to have ceased operations before they had their cases closed. From the App Academy Citation Affirmed decision:
App Academy paid the $50,000 fine, but didn’t close down their bootcamp for months at the beginning of 2016 - at least, not that I can tell.
This story seems pretty typical of the bootcamps that had their citations resolved - they paid a fine (ones I saw ranged between $5000 and $100,000) and BPPE stopped worrying about them.
Often, the final correspondence from the BPPE mentions “Evidence of Compliance”. I haven’t seen the actual evidence of compliance either linked or described in the letters - mostly just confirmation that the fine was paid. Spooky.
Is the mandate that schools close just a threat? Is it just not observed? Seems super weird to me that they’d put it in there, then never once enforce it.
What is regulation for?
The BPPE Mission, from the website:
The Bureau protects students and consumers through the oversight of California’s private postsecondary educational institutions by conducting qualitative reviews of educational programs and operating standards, proactively combatting unlicensed activity, impartially resolving student and consumer complaints, and conducting outreach.
The point is to protect students from predatory institutions, who advertise themselves as a pathway to a better life, but then fail to deliver. Ideally, this regulation would provide maximum protection from students, with the minimum of costs for institutions to comply.
As far as I can tell, most of the schools that BPPE cited were doing okay - decent graduation rates and decent job placement rates. Not perfect, by any means, but each case tells a story of real teaching and learning going on, and students being treated fairly. The schools that ended up closing seemed like they were either outright scams (Coding House) or closed for reasons other than just regulatory action (and might have been towards the lower-quality end of the market). The exception here is Origin Code Academy, where the regulator seems to have played a big role in the school’s closure.
With that exception, it doesn’t look like the ultimate resolution of the regulation had tons of false positives or false negatives. That said - it’s a costly legal headache and PR blow to a school to have regulatory action taken against a school!
For the schools that continue to operate with decent outcomes, there’s a lot of skepticism to overcome. Headlines never go away, and “California regulator seeks to shut down ‘learn to code’ bootcamps” sounds pretty scary! The fines are also nothing to joke about - consider Flatiron’s $350,000 settlement with NY’s Attorney General in 2017.
Adding disclaimers to the job placement numbers doesn’t seem to make things more clear. There is a long-running lack of consistency in how schools report graduation rates and job placement numbers, and it’s not evident that publishing caveat-filled pdfs helps students tell their chances of graduating and getting a job.
Schools shouldn’t deceive students. Predatory schools exist, and it’s not clear from the outside which schools are good and which aren’t. This makes it really hard for prospective students not to get swindled!
The sad thing is that the regulator does not add clarity to the picture. BPPE action is not a reliable signal that a school is a scam. Of the 15 schools whose cases are closed, 11 were eventually approved, and only one of the remaining 4 looks like it was clearly a scam. I would love it if the regulator were a reliable source of truth on this - then, any time I saw a regulatory action, I could trust that the school was actually bad.
My best guess is that Holberton, Lambda, and tradecraft will come out the other side with a story similar to App Academy, Hackbright, or Coding Dojo. They’ll pay a fine, and the school will keep on running with minimal changes, after a long time in legal limbo. If I was a prospective Holberton, Lambda, or tradecraft student, I would take the BPPE action with a grain of salt.
To me, it also seems worth noting that regulation protects large players and incumbents, and acts as a moat against new entrants. If you’d like to see more people starting schools, or see the smaller schools succeed against the big players, you’d be better off without a year waiting for approval before you can start enrolling students. On top of the initial cashflow issues that financing tools like ISAs can present to schools, a slow, opaque, and dangerous regulatory environment will scare off folks who want to start something new.
I think that’s worse for the bootcamp ecosystem.
Notes from reading a ton about the BPPE
There’s a lot of interesting bits and pieces to pick up from browsing the data. Here’s some that struck me (but you are hereby encouraged to browse BPPE’s site yourself!)
There are lots of bootcamps that no longer exist! Origin Code School, Rocket Space’s RocketU, Coding House, and Orange County Code School (as far as I can figure) are just the California schools that had some regulatory action.
There are schools that exist but aren’t listed on Course Report! @codedistrictla, I’m looking at you!
Helpfully, Course Report preserves info on schools that have closed. It doesn’t include them in the list of schools, but since their audience is mostly prospective students, that seems like a good design choice. 👏
LaunchCode didn’t have a regulatory action against it, but did get denied when they applied for approval to operate (link shows a full list of institutions with denied applications). They look like they’re still operating, but never opened in California. Not sure how many others fell into this category, since I stumbled on this one by accident!
Interestingly, different schools have different kinds of owners - some are listed as an LLC, some have the founder listed as owner. Doesn’t seem like it matters a whole lot one way or the other.
BPPE regulates lots of sharks, and maybe ISAs are good
There are a lot of crappy-seeming trade schools in California. Of the ~700 schools with action taken against them, only 18 were bootcamps. It jumped out at me that so many were cosmetology schools. It might be that I was looking for them - I had read the December 2018 NYTimes coverage of cosmetology schools. You should too, if hate-reading about the shadiest parts of the for-profit higher ed ecosystem floats your boat. Key quotes:
Cosmetology schools took in nearly $1.2 billion in federal grants and loans during the 2015-16 school year.
Ashley Sandoval makes $10.50 an hour at another Great Clips location. In the five years since she graduated from cosmetology school, she said, interest has ballooned her debt from $22,000 to $29,000. “I’ll be paying it off for the rest of my life,” Ms. Sandoval said.
These are from schools in Iowa and Nebraska, but at a glance, there are more than 40 citations against schools with ‘beauty’ in the name on the Big List of BPPE Disciplinary Actions.
To be 100% clear: coders are not better than cosmetologists. For-profit coding schools tend to give their graduates high paying careers, and for-profit cosmetology schools tend to saddle their students with debt and no way out.
Just a thought - instead actions against schools that issue ISAs, BPPE could force all job training programs run on ISAs. Student-protecting regulation, like that proposed in the ISA Student Protection Act, can make ISAs better instruments than loans. Key provisions:
time limit on repayment
cap on total cost
income threshold before repayment kicks in
protections against discrimination
ISAs can choke predatory schools. If students have to get a job in their chosen industry and make above a threshold income before they have to make ISA payments, schools that don’t get their students there will disappear - and their students won’t pay for the school’s failure.
State of the Bootcamp Market
Career Karma released their State of the Bootcamp Market Report 2020. The report is worth reading in its entirety.
If you’re new to bootcamp market sizing reports, the Course Report Market Size Study from 2019 and 2018 outcomes and demographics report are also awesome.
Here are the highlights:
33,959 students graduated from bootcamps in 2019. Growth in the market looks like it’s plateauing, with 4.3% YoY growth, compared to the annual doubling from 2012 to 2015.
A lot of the growth in graduates is coming from online - online graduation is up more than 30 percent.
General Assembly is way in front of other bootcamps in terms of total alumni, with 44,800 grads. Trilogy Education, which is behind a lot of the university-branded bootcamps, is next with “over 20,000 graduates”, followed by Hack Reactor / Galvanize with 6,300, and a long tail of other bootcamps.
ISAs are big, and will get bigger. There’s a nice comparison of ISA terms, which will hopefully lead to gradual improvement for students, as bootcamps compete on their ISAs.
The study pulled data from LinkedIn, where CourseReport surveys the bootcamps themselves. Each of these methods can have issues - folks might put anything (or nothing) on LinkedIn, and bootcamps have notorious trouble getting all of their alumni to report back on their actual outcomes (and Course Report doesn’t independently verify the numbers the bootcamps share). Still, it’s a nice source of data to pull from, even if it’s imperfect.
Learning Roadmaps
One of the hardest parts of learning to code is figuring out exactly what to learn.
Particularly when you are just starting out, you don’t know enough to navigate all the potential things that are out there to learn, or recognize which resources are right for you.
There are an overwhelming number of resources for learning. See, for example: 610 Free Online Programming & Computer Science Courses You Can Start This February). Before you actually dive into a resource, whether it’s a tutorial, an online class, or an interactive tool, it’s hard to know whether you’re ready for it, or whether it’s actually something you want to know.
It’s hard to see how the pieces of the puzzle connect to each other. How does learning HTML help with learning Javascript? What about APIs, or algorithms?
Without actually learning each of these topics, it’s hard to see how they’re related. Without understanding those relationships, sequencing the steps in the learning journey is almost impossible!
This environment leads to two big problems for new code learners: thrashing and discouragement.
Thrashing is when you flip between different languages, tutorials, and courses, but make limited or shallow progress on any particular one.
It’s common to hear beginners say things like: “Yeah, I’m learning HTML, CSS, Javascript, Python, Java, and VSCode, and I’m interested in learning Angular and GraphQL and Docker next.” I’ve seen students struggle to make forward progress, especially when they’re ‘researching’ that way! Dozens of tabs open, task-switching, and paying huge costs in cognitive load. This is learn-to-code analysis paralysis.
Given the thrashing, and how much content there is out there to learn, it’s natural to see how a beginner might be discouraged! Not only is progress molasses-slow, but there’s a sense that you’ll never be able to learn it all.
The emotional and self-regulation journey of learning to code is as significant as the skills-and-knowledge journey.
A few links to roadmaps for code learning journeys:
Bento.io Learning Tracks - organizes the learning journey into tracks for different topics, with links to tutorials and time estimates at each step. It’s got a checklist-oriented approach, which helps learners keep track of what they’ve learned, so they can stop thrashing. It’s nicely curated, the sequencing is reasonable, and the recommended things to learn are high quality. This is one of my go-to recommendations for folks who express a general desire to learn coding, but don’t know where to start.
Codementor’s interactive quiz for picking a programming language to learn, which might be helpful for getting a choice made and committing to making progress instead of spending more time choosing.
Frontend, Backend and Devops Roadmaps from roadmap.sh is recently making the rounds on twitter / in newsletters. It has an opinionated set of choices - the author recommends a particular tool from each category, but lists the alternates. It’s nice that it implicitly acknowledges that you shouldn’t learn everything in each category of tool, but can instead just pick one. It’s still pretty overwhelming.
re: that last link,
The tech industry is an intimidating place for people who have merely gotten degrees and worked in it for 20 years
— Camille Fournier (@skamille) February 7, 2020
None of these roadmaps will solve the problem of thrashing or feeling overwhelmed or in learning. There’s still so much here to absorb. Getting out of the thrashing-and-discouragement trap takes recognition that the learning journey will take a long time (even after you’ve gotten a job! imagine that!) and a commitment to making steady progress in one thing at a time, instead of flipping between different tutorials.
I suspect that much of the benefit of a bootcamp, degree, or online class is to remove topics from a student’s radar. A curriculum limits students choices, and forces them to focus on one thing at a time. Schools teach too, but a good chunk of the value is helping students get out of their own way.
Links
For more intimidating-but-interesting things for your never-ending list of things to learn, try:
The Missing Semester of Your CS Education. It’s aimed at filling in the gaps that CS classes miss. In particular, they focus a lot on command line tools for increasing productivity. CS programs have a bad rap for prioritizing the theoretical over the practical, and it’s cool to see students take it into their own hands to fix that. If you’ve already learned some programming, but are a little intimidated by people who are awesome at the command line, this one might be worth a look.
What every Computer Science Major Should Know is another CS-major focused resource. Like many resources in this genre, it lists more topics than most people probably have time to learn. Still, worth actively deciding not to learn those things, rather than not knowing that you don’t know.
If you have favorite roadmaps or concept maps of coding concepts, send them my way!
—
That’s all for this week. Let me know if you have thoughts or feedback. If you enjoyed it, send it along to a bootcamp teacher you know, or drop a link in your favorite bootcamp Slack channel!