Nostr Web Client

Some more thoughts specifically about writing them tests first:

- You might have written the wrong expectation because you’re post-rationalizing the implementation.

- Is coverage branch coverage or line coverage? How often is each line/branch covered?

- Do you properly mock/stub? Is your mock reimplementing the implementation? Is your stub skipping functional parts?

- Did you reimplement parts of the implementation in your tests? This sacrifices resiliency, let‘s say if the factory seed data changes, then all your tests suddenly behave unexpededly.

- Do you have too many tests? Since tests are part of your software design they may also hinder you. Especially if you‘re going up higher in the testing pyramid.

Laeserin 1y ago

- You mean, if you write the test second, then you might write a different test because of sunk costs in the implementation?

- I'm aiming for 100% branch coverage in the unit tests. I don't find that daunting because I tend to otherwise do code-walkthroughs and we're skipping that. Just PRs. So, it feels like I'm doing my own code-review, that I can rerun again and again.

- Stubs can't fail a test, though, right? Isnt't that what differentiates them from a mock?

- No. I just use the implementation, I don't recreate it. I do thinly recreate some things that are outside of the repo, tho, to fake an interface.

- I think this is actually going to reduce the number/complexity of tests I do, or at least, front-load the testing. I usually don't develop anything and most developers refuse to write unit or integration tests, so I'm always stuck trying to figure out ways to test everything through the user interface, which is often really difficult or even impossible. So, I think the tests for this system will be more pyramidal, with 100% coverage at the unit level, 50% at the integration level, 25% at the system level, or so. It's so satisfyingly easy to test things at the lowest level.

Reply to this note

Please Login to reply.

Discussion

Josua Schmid 1y ago

1. No, but that‘s a good point as well. I didn‘t think of that yet. My point was that you cannot unthink the implementation. So you will write tests instead of executable specs. Using specs to lead the design of the implementation is way more powerful. Less confirmation bias involved.

3. Correct. But too agressive stubbing may leave your tests vulnerable to not detecting implementation changes. As well as too heavy mocks.

4. Then you’re disciplined.

An example:

a = 5

b = 2

expect(calc(a,b)).to equal(5 * 2)

I often fall into this trap, because I want the test to be very readable.

Laeserin 1y ago

1. I guess that doesn't really work with me because my specs are BDD-style. I don't write technical specs, just requirements as tests.

2. Oh, okay. I guess I haven't done enough of that, to encounter that problem. They usually give me the thinner stuff to do, like ORMs or clients.

3. Interesting. Must be because I'm a tester, rather than a "real" developer.

Josua Schmid 1y ago

If you say BDD, do you mean Cucumber/Gherkin? I wanted to talk to someone doing this in real life since a long time ago.

Laeserin 1y ago

Yeah, Gherkin. It was in PHP, at my last job, but I want to do some python this time around. We already defined the user stories with AC, to prep for doing this, later.

Josua Schmid 1y ago

What is AC?

I‘d like to start using Cucumber in the context of domain driven design. My problem is that the initial design phase of projects never leaves space for this budget-wise. While I learned in engineering school that 10% of the project budget is to be allocated for an architectural prototype, the situation on our agency projects is completely different. It starts from selling the shiny thing and then underpinning the business logic in a move-fast-break-things manner. We are at least disciplined enough to have 100% unit-test coverage. But how do I make a proper argument for long-living domain logic tests which are readable for the business side and serve as a discussion base for business model design changes?

Laeserin 1y ago

Acceptance Criteria

Laeserin 1y ago

We just do no other specs and use them as our manual acceptance test.

Josua Schmid 1y ago

Do you outsource the programming work? Or are the Gherkin specs used in house?

Laeserin 1y ago

In-house.

Josua Schmid 1y ago

1. Are they written as end-to-end tests?

2. Let‘s say using Selenium?

3. What‘s the abstraction level in practice? Is it „when I click button X…“ or is it „when a salesperson creates a discounted offer“?

Please tell me when I ask to much.

Laeserin 1y ago

Hey, no problem. I'm surprised to find someone interested in this topic. 😄

They're written as business scenarios and you can combine them into E2E tests. You can reuse one scenario like "login with valid customer profile" into lots of different E2E tests.

At my old company, we were using Behat. That was a couple years ago. It's based upon Gherkin and the parsers often use Mink (similar to Selenium) or something. But you can also

https://docs.behat.org/en/latest/

I know some java developers that were using Cucumber Open, I think:

https://cucumber.io/tools/cucumber-open/

Josua Schmid 1y ago

I rediscovered my interest for BDD with the rise of ChatGPT. The long road to „AI“-supported coding must include good and executable specs somewhere, I believe. We‘ll be in an undebuggable mess otherwise.

Josua Schmid 1y ago

What percent of project budget do you typically allocate to writing those business scenarios?

Laeserin 1y ago

About 15%, probably, as you sometimes have to rewrite and you have to make sure that they all fit together.