Do you write tests?

I let the users test it for me, LOL.

For MVPs writing tests for anything that's not critical is a waste of time. At that stage your biggest risk isn't shipping bugs to production, but building something nobody wants.

Once you get paying users, you obviously want to write tests to avoid shipping stupid bugs.

I write tests for two reasons:

1. To tell me that the changes I made resulting in expected outcome
2. To avoid manually testing things

Those two really help speed things up.

The tricky part is to decide when not to test.

Yup, I do write tests. Out of habit, but I think it helps when the features get more involved and more things could break.

It does feel like I’m slowing myself down, but it’s an insurance policy to protect my users from bad coding.

I write tests when it helps development run faster - if I'm writing an algorithm or parser or something like that - any sort of "data in, data out" function, it makes so much sense. Just like Jest in watch mode for testing TypeScript is a really nice way to progress.

Integration tests, though - I've written them, and they're terrible and slow down development. For simpler applications, fine, they work well, but on every project I've ever worked on through 3 jobs, I've never seen them have much of a ROI.

Yes. For Stackless, being a 'business platform' it must be fully tested. However, because I'm a single person writing an entire cloud development platform, I've optimized my use of automated tests so I'm not doubly testing anything except the most low levels of the system.

1. The core, the engine code that interacts with V8 and RocksDb has 100% code coverage using cpptest Unit Tests.

2. The network IO code also has load and fidelity unit tests that run at the same time as the engine.

3. The front-end runtime TypeScript packages (called the SDK) are both smoke tested via an automated suite of tests that ensures all the use cases I've marketed in the public examples work as advertised. Basically, the example applications I've built are tested thoroughly before a new version of the SDK is published to NPM.

Coupled with my penchant for developing defensive code, I believe this strategy has provided me with a happy medium between productivity and risk.

The downside to this approach has been that it's a little tough diagnosing bugs found by the smoke test. But, the extra architectural agility I'm afforded by constraining my direct testing surface area to mostly the "implicit contract" my users expect has been worth the occasional pain.

Yes, I write tests, but at others mentioned, it depends on a number of things:

• The nature of the project. A tool that earns you money is arguably something that shouldn't break. A mature product that will see major refactorings should also be tested

• the code to (not) test. I tend to write unit tests that are simply quick to write (think: utility methods), and tests for user workflows. Another thing is external APIs that may break your app even without you doing anything, so that's something I'd test. And last but not least, I test code that was not trivial to get right, in order to prove that it's correct.

IMO - Basic unit tests are important and may save you time in the future, but be careful because it is easy to create unit tests with a lot of green checks even if your App is not working.

Those bullets should be your top priority, even before you invest time in unit tests:

1. Catch every exception, and add logs to your app.
2. Know your site metrics - for example, If there are Y users on the site with a %X conversion rate. Make sure the numbers are not drastically changed in newer versions.
3. Send your site to friends after every major upgrade. You will be surprised how many things they will find.
4. Use Hotjar (hotjar.com) to track your visitors and make sure everything is working. You can also add a feedback button, but most of the time visitors won't do the effort to tell you about bugs.
5. Pay a few dollars to find someone on Fiverr for the QA. Pay him a bonus if he finds a bug, and trust me - he will.

Advanced (but must):
Write a test that mimics the normal user and that tests the most critical parts/APIs of the site.

Absolutely, 100%. It's 2022, not 2002. I believe tests are making you faster, not slower. I also write all kinds of tests from unit to system depending on what I think I need to cover. However, I never obsess about code coverage tools and numbers.

first test then code

test

Some of the numerous projects I never published I wrote TDD, used fancy new stuff and really wanted to make everything great. However, I promised myself to stay as MVP as possible in the future to validate. Boring tech, ugly code, no tests :)

I write shockingly few tests—indie or day job. I don't claim to write the optimal amount/proportion of tests, but I find that most bugs come from not understanding the problem, the workflow, and/or the edge cases, and if you don't understand those, your tests won't be any better than your application code.

The tests I like most are almost at the extreme ends:

• Cypress browser tests that cover an entire user workflow
• fast, terse tests to cover as many cases as practical of libraries meant to support flexible usage

For indie projects few tests. Mainly I would test an algorithmic thing. Rarely would I test a DB/UI thing. Bang for buck on the algorithmic thing is way higher (more likely to go wrong and way more easier to test)

i dont know much about this, but i am trying this on this website https://thenewsaddicts.com/

In my experience, my meticulousness in writing tests depends on where my project is.

I tend to look at projects on a scale from "no one would use this even if it were free" to "I am the power company" (i.e. I control a thing that no one can live without - I've never actually made a project like this haha).

Before I've validated that people would use it at all (usually pre-beta test), I tend not to write very many, if any tests. Code that no one ever uses is guaranteed not to have bugs :)

But usually if I'm building a project for myself or if I have some beta users who seem to like it, I'll test more complicated logic or utility functions. For instance, one of my projects involved a web crawler that needed to normalize links for pages that may or may not have canonical links in their HTML - I wrote unit tests for that.

More extensive end to end and integration tests require a lot of upfront work, and in my opinion, it isn't worth guaranteeing you haven't broken anything until you've guaranteed that people will actually pay you for your project. Once people are paying you, I believe you have a definite obligation not to break their stuff.

Yes, for most of my applications, except small prototypes/MVPs where I need to quickly validate something.

I'm writing top-level feature and integration tests

I write some E2E and much else, especially pre-launch.

I do love TS, though, I've been writing it since 2016, and it doesn't particularly slow me down; with that said, I wouldn't recommend it to beginners who want to launch something quickly, at least not using strict configurations.

I think a lot of the adversion to writing tests is that devs haven't put together their test stack. Which means that testing and setting up the test infrastructure becomes more of a burden.
https://teatv.ltd/
https://hellodear.in

I think a lot of the adversion to writing tests is that devs haven't put together their test stack. Which means that testing and setting up the test infrastructure becomes more of a burden.

This is totally fair, btw, pushing features is your lifeblood, but id be interested to see if barriers to setting up testing (time cost) which could be addressed by having a more off the shelf testing set-up.

Spend some time researching the test stack for your language and framework to make testing less intimidating :).

Even on MVPs I'll still do testing (obv I'll cut corners) because it feels like a piece of the code that following the Pareto principle puts you in a great spot!

My test stack:
I have a much less mature typescript test stack but here is my RoR off top of my head:

Rspec
Capybara
Factory Bot
Faker
VCR - makes mocking requests a breeze...this was a life changer for me
DatabaseCleaner
Selenium webdriver

It's important to recognise that the more tests you write, the less users you will get, and vice versa ;)

I do write feature tests.

I dont 😂

As someone who's in the validation stage (but also a developer and just barely resisting the urge to start building), I've been thinking about this a lot lately. Because this is a nights/weekends project for me, I keep thinking I should forego tests until I'm at least in beta. On the other hand, I'm not sure I'll be able to actually sleep at night during a beta if I don't have at least some critical paths tested ahead of time.

Recently I've started diving into Cypress (https://www.cypress.io/) and have found it surprisingly easy to properly test with.

For me, you test to give you more confidence in you code and shipping. Early stage with no users, it's something you can easily neglect. But as more people start using the startup, you want to test to avoid those cold sweats when a user flags an issue in your app.

I generally focus on having at least some integration tests for the mainline functionality plus anything that's close to the conversion funnel for my startups.

General answer to general question: yes. Test your shit before customers see it.

More nuanced answer: it depends. My thought process is usually around two pillars:

--

Development speed: will writing tests help me go faster? The answer is often yes, if you structure things a certain way. There are cases where it only slows you down, and it takes experience to understand the difference.

Risk management: what are the likely outcomes if I ship buggy software? Specifically, what's the likelihood of something going wrong, and what is the impact? This really comes down to your market: how sensitive are they to failure; and your software delivery: how quickly can you release a fix?

---

Personally, I test my primary business domain code the most for correctness/documentation (closer to BDD) and development feedback speed, and end up with integration tests on every endpoint I care about.

Huge +1 to leaning into your type system (note: this is a skill you must practice) and removing a large class of tests that make your code fragile.

I left a more detailed reply on reddit, re: QA the other day as well for more general thoughts on QA.

Just because these are start-ups / solo developers, the thought process should be the same. The only thing that changes is risk management trade-offs. Manage them risks!

I wrote a test function that I can turn on and off. I did this because my app has a ton of functionality that was hard to test manually, but easy to test with a simple function.

When I make sweeping changes, I turn on the test function, then turn it off after the changes are pushed live.

TypeScript is great. Type annotations means that you don't need to write tests just to ensure you haven't misspelled a parameter or misremembered how to dig deep into some structure you're using.

The "my code is 50% type annotations" is...FUD. Plain and simple. React does make you write a bit more by way of type annotations than, say, Node.js backend code, but as soon as you get into anything complicated the type annotations will save your bacon many times over, more than paying you back for type-heavy tiny components like the above.

As to testing: I'm simply very selective about what I test. I wouldn't likely have a test for a component like the above, for instance. It doesn't even really make sense to test it in isolation; what are you testing? That the img tag works? I might have tests on larger components or on full pages, but not on every tiny little function.

I've been programming for a very long time. Coming up on 40 years, I think? I have a very good sense of both where bugs are most likely to live and where they're most likely to be a challenge to find. In those places I put a lot of test coverage, as well as runtime asserts and logging if the code encounters unexpected states.

When I've been involved in projects with extensive tests, 99% of the tests seem to simply never fire. Yes you get lots of little green check-marks, which is satisfying. But each one of those checkmarks, in addition to representing a test, also represents technical debt. Any part of your code that's thoroughly tested at a low level is that much more resistant to being modified. If you only have a small number of tests to verify high level functionality, then changing how your app works is actually easier.

Refactoring is easy with static types, too. About 90% of the time I can perform major refactoring, moving a lot of code around and improving it as I go, and at the end it just works the first time after it compiles. The other 10% of the time it takes a minor tweak or two to work.

Using extensive static types in TypeScript or otherwise is a super-power. Dynamic types are a lie: They help you get started at the very beginning and then as the project progresses they add more and more technical debt. You get behind on documenting code (the types in a static-type language represent a minimal required level of known-good documentation); you forget how things are supposed to work because it's not all documented in one place and so structures grow in awkward ways; code that you thought was obvious at the time now takes you an hour to figure out before you can add that feature, where if the types were specified everywhere you could just have made the change in 10 minutes; the list goes on and on.

I have a bad habit of not writing tests until I start to get annoyed at myself for regressions.

And then by that time I've realized I built something no one wanted, so I'm glad I didn't write tests...

Nop.

It really depends on what the product is. If the product is straight-forward enough, writing tests for the very first release might be a lot of wasted time on your part, especially if you have zero users and expect to change everything in the near future.

Once you:

1. Know what you want to go forward with
2. Have people using your product
3. Expect that the product will eventually become complicated

...then you might want to look into writing a good test suite to make future updates safer. Otherwise I'd say it's more optimal to spend this time developing the product itself. I'm doing exactly that with my current product; I shipped an initial release with zero testing, and now that I'm starting to have users, I'm considering taking a step back and safeguarding it before continuing.

Maybe I can do

TypeScript is great, but type safety does not necessarily guarantee runtime safety.
I tend to write tests regularly, they are another tool for me that gives me confidence to refactor stuff.

On the other hand, tests are the perfect tool for me to debug things.
A test is the perfect setup for debugging.
You have a fixed set of input and output values, paired with an easy to execute, reproducible setup.
Makes stepping through code really easy!

Last but not least, every time I encounter an issue in production, caused by an edge case or a regression, I write a test for it to confirm it's still fixed in future releases!

EDIT: I forgot one really awesome thing. A meaningful set of tests is pure gold when searching for a broken commit using git bisect

As a general rule of thumb, I would only write tests that can negatively impact users. This means that having users who actively use the product is a prerequisite to spending any time on test automation.

To decide what features to test and how, let's take my product, Makery as an example.

1. Identify critical features
   Makery is a static site builder, so updating websites and publishing the changes can be considered critical. I host Makery's landing page on Makery and I make changes quite frequently. This means I basically manually test the site update process every time I publish something new.

2. Pick features that you don't cover usually by yourself
   I only have one website, which means I create new websites less frequently. Website creation actually involves provisioning new resources in third-party services, saving new records, creating build artifacts, and publishing them online. I would consider this use case a good example to write a test for.

3. Optimize for customer value
   The best tool to pick usually depends on everyone's experience and background, so there isn't any good or bad choice here. One important thing to consider is that you want to optimize for customer value and speed, not for technical challenges.

So if I get back to my example, my goal is to make sure that creating a new website works. So it is completely fine to add let's say a Cypress test that clicks the Create website button before every release, even if once in a while I have to manually delete a couple of resources.

As the tweet says, Typescript is awesome! I use types extensively to pin down invariants in my code to let the type system do the heavy lifting. Types replace all the 'defensive' unit tests one would otherwise have to write in JavaScript to feel comfortable.

As far as what I do actually test for my pre-MVP project, I write tests for the little utilities and types to make sure they work (these are small unit tests). Otherwise I'm doing manual testing and any gaps I find there I look to incorporate into the type system.

I'm writing code that can be tested up front. I definitely have integration, E2E, and property-based testing on my radar as I write any code. I will want that eventually.

I write backend tests (not familiar with frontend testing). I try to keep coverage above 85% and ideally over 95% if the product gets more validation. I do this for MVPs as well because it's easier to spot errors in my code -- if I go back to the codebase after a while, tests also help me understand the default behavior + edge cases easier.

Yes, and I have a CI/CD pipeline setup as well. I want to push my code, and focus on other things, while I know the testing of critical features is being automated.