Chapter Roadmap (1 of 2)

• 15.1 What an assistant gives you, and who stays accountable
  • The six-step workflow behind every AI activity in the book.
• 15.2 Telling the assistant what you want: task and context
  • A prompt is a work order, not a requirement.
• 15.3 Asking an assistant to explain: requirements and code
  • Classify explanations claim by claim.
• 15.4 Candidate code and tests: a worked task
  • The worked task: two rejected attempts and an accepted version.
• 15.5 Asking for a refactoring or a review
• 15.6 How assistants fail: the patterns to recognise

Chapter Roadmap (2 of 2)

• 15.7 Privacy, dependencies and where work came from
• 15.8 Writing down what you checked: verification records
  • What evidence can and cannot show; a compact eight-part record.
• 15.9 Doing all of this without a live service
• 15.10 AI habits that mislead
• 15.11 Try it yourself: the lab and your own task
• 15.12 Questions to think with, and a transfer challenge
• Summary and where to go next
  • The appendices collect what the chapters point to.

Learning Outcomes

After this chapter, you should be able to:

1. Provide bounded context for an AI-assisted task: what it is, what it may use, what it must not do, and what counts as done.
2. Evaluate candidate explanations, code and tests against evidence, claim by claim.
3. Recognise unsupported APIs, invented methods, unnecessary abstractions and tests that repeat the code’s mistakes.
4. Record useful provenance and verification evidence.
5. Complete an equivalent exercise without live AI access.

Where This Chapter Fits and Where You Start

• This is a reference chapter that belongs to no teaching week. It supports the module’s outcomes wherever an assistant is used and adds no separate assessed AI outcome.
• It creates no new deadlines, weights or marking rules. Whether, and how, AI tools may be used in an assessment follows the module leader’s current rules.
• Entry version: none. The worked example is a small separate project, the lab, that stands alone, compiles for Java 17 and needs only the pinned Maven toolchain of the earlier chapters.
• The lending project is not changed: its handover version 0.14.0 stays as Chapter 14 left it.
• The chapter is vendor-neutral. It does not teach prompt tricks, model internals, retrieval or agents.

How to Use This Chapter

• Core: 15.1, 15.2, 15.4, 15.6 and 15.8.
• Working knowledge: 15.3, 15.5 and 15.7. Learn the method; you do not need to memorise tools.
• What you must do: B1 to B4 (the book’s project, the lab: running, tracing and diagnosis, everything supplied) and T1 to T3 (your team’s project: apply the method to your own work).
• These steps consolidate what the “AI in practice” activities of Chapters 1 to 14 already ask. Do B1 to B4 once, where you first need them: alongside the first AI activity, or before your final handover.
• If you want more: B5 to B7 and T4, the questions and the transfer challenge. Nothing required depends on a paid account.

Opening Problem: The Confident Note

It is late. You ask an assistant for a small method, and it returns tidy code with a confident note: “I tested this with Java 17 and it works.”

1. It compiles on your laptop. The tests it wrote pass. You commit.
2. The next morning the pipeline fails to compile: one call exists only from Java 21.
3. A reviewer asks why the tests agree with the code when your teammate’s examples do not.
4. Another asks where a dependency named in a pom.xml snippet came from, because nobody can find it.

What Was Missing

• None of this needs a bad assistant. Each needs only an ordinary one and a missing habit.
• The habit: treating the output as a candidate that someone else’s evidence must accept or reject.
• This chapter collects that habit into a few workflows you have already practised in Chapters 1 to 14, shows them end to end on one small task, and says what to write down so the work can be reviewed.
• AI-assisted development (an assistant helps you build software) is the topic. Software that contains an AI component, such as the borrower directory, is treated where it arises and, for a model, in Appendix F.

Before You Start: Five Rules (1 of 2)

1. You are accountable. The assistant proposes. You decide, and you answer for what you commit.
2. The requirement is yours. An assistant can ask good questions about it. It cannot decide what the equipment office wants.
3. Keep the evidence independent. Check the output against something the assistant did not write: the requirement, examples you wrote first, the compiler, a trace by hand. Another assistant’s opinion is not independent evidence.

Before You Start: Five Rules (2 of 2)

4. Share the minimum. No secrets and no personal data. Use synthetic data.
5. Follow the current rules. Whether a given tool may be used on coursework or in an assessment is decided by the module leader’s current rules, not by this book.

What an Assistant Gives You

• An assistant can explain code, suggest an implementation, propose tests, point out risks in a change and draft documents.
• Each is a candidate contribution: plausible text or code that may or may not be right.
• GitHub’s documentation for one such tool says suggestions can look valid and still be incorrect, and the user must review and validate them before accepting. Users assume the risks of generated code: bugs, security flaws, intellectual-property questions.
• That is one vendor’s statement about one tool. The discipline it describes is general.

Where the Decisions Stay

The Assistant’s Statement About Itself

• “I tested it with Java 17” is a sentence, not a test.
• The assistant’s statement about itself is a claim like any other.
• Treat every such statement as something to check: an assistant that cannot run your build cannot have tested it under your settings, and one that can run something still ran what it chose.

A Workflow in Six Steps

Reading Figure 15.1

• The first box is yours: derived from the requirement, before any candidate exists, and it is the authority the checks refer to.
• The loop from the checks back to the task is where most of the work happens.
• A candidate that fails is information. Keep it.
• Every AI activity in this book has this shape.

The Activities You Have Already Done (1 of 2)

The Activities You Have Already Done (2 of 2)

• The checks vary, and none of them is “ask another assistant”.

A Prompt Is a Work Order

• A work order says what to do, with what, within what limits and in what form. It is useful, and it is not a substitute for the requirement.
• If the requirement is unclear, the assistant fills the gap with a guess, and a fluent guess looks like an answer.
• Keep the context small. More text is not more help, and every extra file is something the assistant may use and something you may have shared that you should not have.

The Five Parts of a Work Order (1 of 2)

The Five Parts of a Work Order (2 of 2)

The First Prompt of the Lab

You are helping with a small Java task in an existing project.

Task: add a class `AvailabilityFinder` in package `assist` with a method
`Optional<LocalDate> earliestStart(LocalDate from, int days, int searchDays)`. It returns
the earliest start date, on or after `from` and within `searchDays` days, on which an item
could be reserved for `days` days, given the periods for which the item is already booked.

Constraints: Java 17, standard library only, no new dependencies.
Output: the complete source of `AvailabilityFinder.java` and nothing else.
Scope: do not change `LoanPeriod`. Do not reserve anything.

Derive the Examples From the Requirement First

• Before you prompt, write concrete examples of the behaviour you expect, with inputs and results. Derive them from the requirement and not from any code.
• In the lab they are nine tests written before anything was requested.
• What makes them independent evidence is how they were derived, not whether the assistant saw them.

Should the Examples Be in the Prompt?

• A choice with a cost either way.
• Sharing reviewed examples clarifies the task and often improves the output; they remain useful acceptance checks. But a pass on examples the assistant has seen shows only that it satisfied those examples, not behaviour beyond them.
• In the lab the examples were not shared, so they could be used both ways: as acceptance checks, and to judge whether the code generalises.
• The point is not that hiding examples is the rule. It is that the examples came first, from the requirement.

A Workable Practice in Three Parts

1. State the requirement precisely in the prompt. Share reviewed examples if they help.
2. Add boundary cases or properties that you derived independently, and keep some of them undisclosed when you want to judge whether the code generalises.
3. Never derive the expected results and the implementation from the same unexamined assumption. That is what happens when an assistant writes the code and then its tests from that code.

Ambiguity Is Yours to Remove

• The lab’s first prompt says the search is “within searchDays days”. That can mean:
  • “up to and including from + searchDays”, or
  • “searchDays candidate start dates, starting with from”.
• The developer meant the second. The assistant chose the first, and its tests chose the same, so the code and the tests agreed with each other and disagreed with the developer.
• The acceptance examples were the only thing that did not share the misreading. The requirement sentence was rewritten afterwards.
• When an assistant guesses wrong, ask what in your requirement allowed the guess.

Explanations Are Lists of Claims

• An assistant’s explanation of code is a list of claims, some about what the code shows and some about things the code does not show.
• Break it into single claims and classify each:
  • Supported: the code, a test or a document shows it. Name the evidence.
  • Contradicted: the code shows the opposite. Name the evidence and write the correct statement.
  • Not established: the code does not say. Say what would be needed.
• A useful prompt asks for exactly that separation, and it still needs checking.

A Constructed Explanation of VisitorGate.rejection

rejection first checks the feature flag and returns the “not enabled” text if visitors are off. It then checks that the item is CAM1 or CAM2. If the number of days is below one it returns an empty result without asking anyone, because the count and duration policy will reject it. Otherwise it reads the clock once, asks the approval source for the exact request, and rejects unless the record found matches the request and is valid at that instant. The gate caches the record for later requests and writes each rejection to the audit log. If the approval source throws, the gate returns the “no valid sponsor approval” rejection.

• Eight statements, mixed. Three are wrong. Which?

Checking the Explanation (1 of 2)

Checking the Explanation (2 of 2)

Why the Wrong Claims Are the Dangerous Ones

• The incorrect ones are the most believable, because they describe things a gate like this might well do.
• The third would have produced a wrong design decision about error handling: it is the difference between a rejection and a failure that Chapters 12 and 13 insist on.
• Reading the code, a test and the class settles each in a minute.
• Keep the accurate parts and correct the rest. Do not throw away an explanation because one sentence is wrong.

The Corrected Explanation

rejection returns an empty result when the request may go on to the coordinator, and a rejection text otherwise. It checks the flag first, then the item, then returns empty for fewer than one day (the policy rejects it). Otherwise it reads the clock once, asks the source for the exact request and rejects unless the record matches and is valid at that instant. It keeps no state and writes no log. If the source throws, the exception reaches the caller; nothing is reserved.

Requirements Work

• An assistant can help you find gaps in a requirement: what happens at the boundary, what is not said, which cases conflict.
• Use it to produce questions, and take the questions to whoever owns the requirement. Do not let it supply the rules.
• Chapter 3’s exercise on a story and Chapter 11’s suspected defects had the same shape: the right output of an assistant was a question for an owner, not an answer.

The Worked Task

• Given the periods for which an item is already booked, return the earliest start date on or after from, among searchDays candidate start dates beginning with from, on which it could be reserved for days days.
• Periods are half-open: a booking that ends on a date does not block a new one that starts on that date.
• Nothing is reserved. Invalid days or searchDays (below one) is an IllegalArgumentException; a null from or bookings list is a NullPointerException.
• The developer’s nine acceptance examples EX1 to EX9 were written before prompting.

Acceptance Examples EX1 to EX5

Bookings: 20 to 22 and 24 to 26 October (half-open) unless stated.

Acceptance Examples EX6 to EX9

One Acceptance Example in Code

/** CAM1 is booked for [20, 22) and [24, 26). */
private static AvailabilityFinder cam1() {
    return new AvailabilityFinder(List.of(period(20, 22), period(24, 26)));
}

@Test void ex4_theWindowCountsCandidateStartDatesAndNotDaysAfterFrom() {
    assertEquals(Optional.empty(), cam1().earliestStart(oct(20), 2, 2));
}

@Test void ex5_theLastCandidateInTheWindowIsIncluded() {
    assertEquals(Optional.of(oct(22)), cam1().earliestStart(oct(20), 2, 3));
    assertEquals(Optional.empty(), cam1().earliestStart(oct(20), 2, 2));
}

Attempt 1: The Constructed Response

• Author-written fictional AI output, constructed to illustrate. Not a recording of a real model.
• The response claims: “I tested it with Java 17 and it works” and contains:

List<LocalDate> free = from.datesUntil(from.plusDays(searchDays))
        .filter(start -> booked.stream()
                .noneMatch(b -> b.intersects(new LoanPeriod(start, start.plusDays(days)))))
        .toList();
return free.isEmpty() ? Optional.empty() : Optional.of(free.getFirst());

Attempt 1: Rejected by the Compiler

Compiled with the project’s baseline, --release 17, it fails with two errors:

AvailabilityFinder.java:18: error: cannot find symbol
  symbol:   method intersects(LoanPeriod)
  location: variable b of type LoanPeriod
AvailabilityFinder.java:20: error: cannot find symbol
  symbol:   method getFirst()
  location: variable free of type List<LocalDate>

• Run it yourself: bash scripts/try-attempt.sh 1, or javac --release 17 by hand.

Two Different Failures

• intersects is an invented method. LoanPeriod has overlaps; intersects is what a similar class elsewhere might have.
• List.getFirst() exists in Java 21 and not in Java 17. It is an API newer than the baseline.
• Compiled with plain javac on a JDK 21 or later and no --release, the second one compiles. In the experiment plain javac reported only the first error; javac --release 17 reported both.
• If your only JDK is 17, both commands report both errors: the project’s command is the one that matters, and it fails as it should.

What --release Changes

• --release restricts the API as well as the language level to the named release. --source and --target alone do not.
• The project’s POM sets it to 17 through the compiler plugin’s release parameter.
• The code “works on my machine” and breaks on the baseline.
• The rule: compile and test with the project’s own commands, never with the assistant’s assurance or your own default settings.

Attempt 1 Under Different Settings

• After the invented method is fixed, the code compiles where it was written and fails on the baseline.

A Style Point, Not a Defect

• The code builds a list of every free date just to take the first one. It is correct in spirit and does more work than needed.
• That is a style point, not a defect, and it is the reason to read candidate code and not just run it.

Attempt 2: Compiles, Passes Its Own Tests, Still Wrong

• The second work order names the two compiler findings and asks the assistant to keep the solution small and to write tests.
• The constructed response: “The finder now delegates to an AvailabilityRule, so new rules can be added without changing it (open/closed principle). I added seven JUnit 5 tests and all of them pass. The behaviour matches your description.”
• It compiles. Its seven tests pass. Run the developer’s acceptance examples:

ex4_theWindowCountsCandidateStartDatesAndNotDaysAfterFrom: expected: <Optional.empty> but was: <Optional[2026-10-22]>
ex5_theLastCandidateInTheWindowIsIncluded: expected: <Optional.empty> but was: <Optional[2026-10-22]>

The Loop Is One Step Too Far

• for (int offset = 0; offset <= searchDays; offset++): the window examines searchDays + 1 candidates.
• From the 20th with a window of 2 it examines 20, 21 and 22 and returns the 22nd.
• One of the assistant’s own tests is called theSearchWindowIsInclusiveOfTheLastDay and asserts exactly that.
• Run it against the correct implementation and it fails.
• The generated tests were written from the generated code, so they confirm what the code does and say nothing about what it should do. “All tests pass” was true, and it was evidence about the wrong thing.

Attempt 2, Step by Step

An Abstraction Has a Price

• Attempt 2 has two constructors, one taking a List and one taking an AvailabilityRule, so a bare null fits both: new AvailabilityFinder(null) became ambiguous, and the developer’s own test for a null argument no longer compiled.
• The one-line cast used to run the examples changes only the argument’s type, not what the test asserts. Editing a test to suit the code is the wrong direction in general; here it is only so the logic failure can be seen. The right response is to remove the unneeded overload.

Size of the Abstractions

• The interface has one implementation, built in one place by a factory called once. The task had one rule.
• The assistant called it the open/closed principle. Chapter 5 said when that principle earns its cost; here nothing needs to be extended.

Why the Acceptance Examples Were Independent

• They were derived from the requirement before any candidate existed, and not from code.
• They were also not shared with the assistant, which let them be used a second way: to judge whether the output generalises.
• Sharing them would have been legitimate. It would probably have produced a correct loop, and the pass would then have shown only that the assistant satisfied those examples.
• The prompts show the point: prompt1.md says “within searchDays days” and nowhere says what that counts; prompt2.md asks for tests, which is where the shared misreading gets locked in.

Revision and Acceptance

• The developer wrote the final version by hand, using attempt 2 only as a reminder of the loop shape, and rewrote the requirement sentence that had allowed the misreading.
• The final class validates its arguments, copies the bookings so the caller’s list cannot change it, and calls LoanPeriod.overlaps.
• It passes the nine acceptance examples. It does not reserve anything and does not touch the lending system.
• mvn --batch-mode test ends with Tests run: 9, Failures: 0, Errors: 0, Skipped: 0.

The Accepted Finder

public Optional<LocalDate> earliestStart(LocalDate from, int days, int searchDays) {
    Objects.requireNonNull(from, "from");
    if (days < 1) throw new IllegalArgumentException("days must be at least 1");
    if (searchDays < 1) throw new IllegalArgumentException("searchDays must be at least 1");
    for (int offset = 0; offset < searchDays; offset++) {
        LocalDate start = from.plusDays(offset);
        LoanPeriod wanted = new LoanPeriod(start, start.plusDays(days));
        if (booked.stream().noneMatch(wanted::overlaps)) return Optional.of(start);
    }
    return Optional.empty();
}

• The constructor does List.copyOf(Objects.requireNonNull(booked, "booked")).

Figure 15.2: The Worked Example

Reading Figure 15.2

• Both rejected attempts are kept in the record, with the reason for each.
• Attempt 1 failed to compile; attempt 2 compiled and passed its own tests but failed independent examples.
• The route that worked: the developer’s version, informed by the failures, checked by examples written beforehand.

Do the Tests Detect Faults?

• Ten single faults were applied to the accepted finder. Each was run against the developer’s acceptance examples and, separately, against the assistant’s generated tests.

The Ten Faults (1 of 2)

The Ten Faults (2 of 2)

Why Two Faults Survive

• With days = 0 the finder’s own check is gone, but LoanPeriod’s constructor throws IllegalArgumentException on a period that does not begin before it ends.
• With a null from the early check is gone, but from.plusDays throws NullPointerException.
• Each example asserts only the exception type. The faults change where the exception is raised and not what a caller sees: they are equivalent, and cannot be told apart from correct code by any test of the exception type.
• The generated tests miss the very fault the assistant made, a window one step too long, because they were built to agree with it.
• Ten faults chosen by the author on one small class say nothing about assistants in general.

Ask for a Review as Questions

• Assistants are good at producing a list of things worth looking at. They are poor at telling you which of them matters here, and a review that ends “looks good” settles nothing.
• Ask for questions and risks, and decide for yourself:

List the behaviours of this method that a caller could depend on, and the inputs for which the behaviour is not obvious from the code. Do not suggest changes.

A Refactoring Must Say What Must Not Change

• Chapter 11: a refactoring preserves intended observable behaviour.
• A prompt for a refactoring should say so, name the boundaries that matter and keep the tests out of its reach: “do not change the tests; if a test fails, report it and do not edit it”.
• When a patch arrives, apply its claims one at a time in a scratch copy and run the suite after each.
• A patch that makes three claims and fails one test does not tell you which claim was wrong. Applied piece by piece, it does. Chapter 11’s patch changed a boundary, a lookup order and a default at once.

What a Review Can Check

• A reviewer can check that a dependency points the right way (the compiled dependencies of Chapter 10), that a boundary is covered (Chapter 13), that a number in a document matches a run (Chapter 14), or that an explanation is consistent with the code.
• These are checks someone can repeat.
• “The assistant reviewed it and found no issues” is not one of them.

The Failures This Book Has Met

• These are failures the book has met, with where. Recognising the pattern is quicker than diagnosing each instance.
• Eleven patterns follow, each with a check that exposes it.

Failure Modes (1 of 3)

Failure Modes (2 of 3)

Failure Modes (3 of 3)

Two Worth a Sentence More

• Silent policy change is dangerous because the code can be better, and the commit is no longer a refactoring: reviewers cannot tell which failures were intended.
• Self-reported verification is a special case of confident wrongness. An assistant that cannot run your build cannot have tested it under your settings, and one that can run something still ran what it chose.

What Not to Send

• With a remotely hosted tool, a prompt leaves your machine. Whether and how the service keeps or uses it is in its own terms, which differ and change: read them, do not assume.
• A tool that runs locally may keep everything on your machine, but check how it is actually deployed: a local front end may still call a remote service.
• A rule that works everywhere: send the minimum. No passwords, keys, tokens, personal data, real student or borrower records, or other people’s confidential material. Replace identifiers with invented ones.
• Whether a tool may be used with course material is for the module leader’s rules and your institution’s policy.
• OWASP’s list of risks for language-model applications names sensitive information disclosure.

A Prompt That Should Not Be Sent

Our reservation service is failing for Layla Hassan (student ID 2023-48817, [email protected]). Here is the config that fails; the directory password is in it: lending.directory.url=http://10.20.30.40:8080/borrowers, lending.directory.password=not-a-real-password-just-an-example. And here are the last 200 lines of our production log, which include other borrowers’ names. What is wrong?

• Constructed example: all names, identifiers and the key are invented.

What the Prompt Discloses

The Same Task, With the Minimum Context

A Java client calls GET <base>/borrowers/<key> and sometimes throws HttpTimeoutException after 1000 ms. Here is the method (no secrets, no real URL). Here are the three log lines that show the exception, with borrower keys replaced by B1, B2, B3. What are the possible causes? List them without choosing one.

• It is smaller, safer and more useful: it names what is wrong and nothing else.
• It states the symptom precisely (a timeout after 1000 ms) and asks for causes you will check, not for a verdict.

Verify Every Dependency

• An assistant may suggest a library that does not exist. Researchers who tested sixteen code-generating models on 576,000 samples found hallucinated package names averaged at least 5.2% for the commercial models and 21.7% for the open-source models they tested, and noted that an attacker could register such a name with malicious code.
• The percentages describe that study and its models, not yours. The habit is what matters.

The Dependency Checks (1 of 2)

The Dependency Checks (2 of 2)

• A status of 200 shows only that something is published under that name.

A Constructed Dependency Suggestion

Add these to your pom.xml and use Mocks.firstOf(list): org.mockito:mockito-fluent:5.20.0 and com.google.guava:guava:33.0.0-jre, both with scope test.

• Task given: “Add a test helper that asserts the first element of a list.”
• Status codes read from Maven Central on 13 October 2026. Nothing was downloaded.

The Decisions

• mockito-fluent: reject. It does not exist under that group, and a plausible name is not a reason to search for a similar one. Even a real mocking library would be wrong: a helper that asserts the first element of a list does not need mocking.
• guava:33.0.0-jre: reject. It exists, and nothing in the task needs it: list.get(0) and assertEquals do the job.
• One suggestion was invented, and one was real and unnecessary. Both are failures, of different kinds.

Provenance and Disclosure

• Provenance is where a piece of work came from. For AI-assisted work: that an assistant was used, what for, what you gave it, what you took from it and what you changed.
• It matters because a reviewer or a successor needs it to judge the work, and because your institution may require it.
• Say so plainly and follow the module leader’s rules and your team agreement. Whatever the rules, keep a record of tasks that mattered.
• Questions of licence and ownership of generated code are unsettled and depend on the tool, the jurisdiction and the terms. This book gives no legal advice.

What Evidence Can Show (1 of 2)

What Evidence Can Show (2 of 2)

• Do not use one assistant to approve another’s work and call the result verification. It is a second candidate opinion with the same weaknesses.

A Compact Record: Eight Parts (1 of 2)

A Compact Record: Eight Parts (2 of 2)

• The lab holds a template, a filled example, and scripts/check-record.sh that checks every heading is present and not empty. It cannot tell whether what is written is true: a reviewer must. Appendix C (C.13) has the blank record and a worked example.

A Useful Record, Not a Ritual

• The ritual version says “reviewed and verified” and nothing else.
• The useful one says:
  • attempt 1 failed to compile under --release 17 with two errors;
  • attempt 2’s seven generated tests passed while two acceptance examples failed;
  • ten faults were applied.
• Write the record while you remember, and write it for someone who was not there.

The Filled Record: Task, Tool, Context

• Task: add AvailabilityFinder.earliestStart(from, days, searchDays). The requirement and its nine acceptance examples were written by the developer before any prompt. The developer owns the requirement.
• Tool and model: None: constructed example (author-written, not a recorded service response), 13 October 2026.
• Context given: prompt 1 (task sentence, LoanPeriod source, constraints, output format, scope); prompt 2 (the two compiler findings, the request to keep it small and add tests). Not shared: the acceptance examples.

The Filled Record: Output and Decision

• Output received:
  1. Attempt 1: rejected. It did not compile: LoanPeriod.intersects does not exist and List.getFirst() exists only from Java 21.
  2. Attempt 2: rejected. It compiled and its seven generated tests passed, but it added an interface and helper class the task did not need, its loop examined one candidate too many, one generated test asserted that behaviour, and it made new AvailabilityFinder(null) ambiguous.
• Decision: neither attempt accepted. The developer wrote the final version by hand, using attempt 2 only as a reminder of the loop shape.

The Filled Record: Changes, Verification, Limits

• Changes made: removed the interface and helper; fixed the loop to offset < searchDays; validated arguments; copied the list; discarded the generated tests; rewrote the requirement sentence.
• Verification: javac --release 17 on attempt 1: two errors; attempt 2 with its own tests: 7 pass, with the acceptance tests: EX4 and EX5 fail; final version: 9 pass; ten faults: eight caught, two equivalent. Independent of the assistant’s claims: the compiler, the acceptance examples written beforehand, the fault experiment.
• Limits: the examples are the developer’s reading of the requirement; compiled with --release 17 but run on a newer JDK, so no Java 17 runtime run is claimed; not connected to the lending system.

Every AI Activity Can Be Done Without a Live Service

• The supplied candidate. Each activity includes a constructed candidate output, labelled as such. The lab’s two attempts, the VisitorGate explanation, the privacy example and the dependency suggestion are all supplied.
• Swap with a classmate. Each of you writes a plausible candidate for the same small task, deliberately including one or two flaws of the kinds in 15.6, and keeps a private list. Swap, find the flaws, compare lists. A flaw you did not find is as instructive as one you did.

More Offline Routes

• Write both sides. Write the acceptance examples first, then a quick, careless implementation as if you were the assistant. Run the examples. This shows how an implementation that looks right can fail on a boundary.
• Use a live assistant if you may and can. Only synthetic data and files you may share. Keep the actual output, including the attempts you reject. Treat its account of itself as a claim. Follow the module leader’s rules on assessments.
• Constructed output is not a recording of a real model. It is written to illustrate a failure mode and should be labelled each time. Where a real transcript is captured, record the tool, the model and the date.

Mistakes 1 to 5

Mistakes 6 to 10

Diagnostic Alternatives

• “Reviewed by another assistant” under Verification: a second candidate opinion, not independent evidence.
• Accepting it because it compiled: attempt 1’s second error showed compiling is not enough, and attempt 2 compiled and was still wrong.
• Tests derived from the implementation: theSearchWindowIsInclusiveOfTheLastDay agrees with the code and fails on correct code.
• A record that says nothing: headings present, Verification reading “reviewed and verified”, Limits empty: the script flags empty headings, and a reviewer must flag empty claims.
• A prompt that carries the requirement and the secrets: the privacy example.

What You Start With, and the Demonstration

• You start with: the lab (Companion, Section 3), with its prompts, two constructed attempts, acceptance tests and record. B1 to B7 use the book’s project (the lab); T1 to T4 use your team’s project.
• Instructor demonstration: compile attempt 1 with javac --release 17 and, if a JDK 21 or later is available, once more with plain javac; ask the class why the results differ. Then run attempt 2 against its own tests, and against the acceptance examples.
• Guided practice: take one claim from the constructed VisitorGate explanation and classify it, naming the evidence. Take one statement an assistant has made about itself and say how you would check it.

The Book’s Project: B1 and B2

• B1. Reject attempt 1. Run bash scripts/try-attempt.sh 1. Compile by hand with javac --release 17, read both errors; with a JDK 21 or later, compile once more without --release and compare. List each defect and its category from 15.6, and say what the claim “I tested it with Java 17” can and cannot mean.
• B2. Diagnose attempt 2. Run its own tests (try-attempt.sh 2 generated), then the acceptance examples (try-attempt.sh 2 adapted). Find the failing line, name the shared misunderstanding between the code and its tests, and explain why the acceptance examples were independent evidence. Run the generated tests against the accepted finder and say what the one failure shows.

Expected Output of the Lab Scripts

The Book’s Project: B3 and B4

• B3. Make the tests bite. Apply three faults of your choice to the accepted finder in a scratch copy. Predict which acceptance examples fail, run them, compare. Find one fault that survives and say why.
  • Examples: offset < searchDays - 1, start.plusDays(days - 1), allMatch.
• B4. Fill in a record. For a different, very small task of your own (a method you ask an assistant, or a constructed candidate from a classmate), complete a record using the template and run the checker.
  • A good record uses all eight headings, keeps rejected output, names at least one independent check and at least one limit. A weak one says “reviewed and verified” under Verification.

Your Team’s Project: T1 to T3

• T1. Define one AI-assisted task before you prompt. A small, real task in your project: the requirement, the acceptance examples (derived from the requirement, before any candidate), the constraints and the scope. Decide which examples you will share and which you will keep undisclosed, and why.
• T2. Run it and keep everything. A live assistant if the rules allow, or an offline route (15.9). Keep each attempt, including the ones you reject. Compile and test with the project’s own commands.
• T3. Record it. Fill in the record, including the independent checks and the limits, put it with the change, and run the record checker.

If You Want More: B5 to B7 and T4

• B5. Repeat the lab task with a live assistant, if you may. Which failure modes did it show? An assistant that gets it right the first time is a valid result: record that and the checks that showed it.
• B6. Verify two dependencies. Look up the constructed suggestion’s coordinates, report the status of each and decide what to do about each.
• B7. Swap and find. The classmate exercise of 15.9. Missing a flaw of the invented-method kind is common and instructive.
• T4. Agree your team’s practice. A short note: which tools may be used (following the module leader’s rules), what may be shared with them, how work is disclosed and recorded, and who reviews AI-assisted changes.

What You Should End Up With, and When You Are Done

• What you should end up with: B1 to B4; for your team’s project, one defined task with its acceptance examples, the attempts, a record, and a team note if you did T4.
• Acceptance:
  • the acceptance examples were written before the candidate was requested;
  • every claim the assistant made about itself was checked or marked unchecked;
  • the check used the project’s own compiler settings;
  • a rejected attempt is in the record with its reason;
  • the record’s limits say what was not run;
  • no secrets or personal data were shared;
  • no live tool was used where the rules forbid it.
• Optional: do the same task twice, once with the acceptance examples in the prompt and once without, and compare.

Concept Questions

1. Why is “I tested it with Java 17” not evidence? Name two checks that would be.
2. An assistant writes a function and its tests, and they pass. Why does that not establish that the function is right, and what would?
3. What does the --release option change that --source and --target alone do not?
4. A dependency suggested by an assistant exists on Maven Central. What have you learned and what have you not?

Diagnosis Questions

5. An explanation of a method says “the method caches the result.” The code has no field. What are your options for settling it, and what would you write down?
6. A refactoring patch makes four claims and one test fails. How do you find which claim caused it, and why is the patch’s own summary not the way?

Guidance: Questions 1 to 4

1. It is a sentence from a system that did not run your build and names no command or result. Evidence: compile and test with the project’s own command (mvn --batch-mode test, which sets release 17); run javac --release 17 on the file.
2. Tests derived from the code agree with it, including where it is wrong: theSearchWindowIsInclusiveOfTheLastDay passes on attempt 2 and fails on correct code. Examples written from the requirement beforehand, and a fault experiment on the tests, would show it.
3. --source/--target set the language level and class-file target. --release also restricts the API to the one that release defines, and cannot be combined with them.
4. You have learned that something is published under that coordinate. Not that it is the library you meant, safe or licensed for your use, suited to your build, or needed.

Guidance: Questions 5 and 6

5. Options: read the class for a field that holds a result (there is none); look for a test that depends on caching, and one that depends on its absence (theFileIsReadAfreshOnEveryCall); trace two calls by hand. Write down the claim, the evidence and the verdict: contradicted, with the line or test.
6. Apply the claims one at a time in scratch copies and run the suite after each: the failing claim is the one whose copy fails. The patch’s summary is a claim of the same kind as the code and cannot tell you which of its own claims was the cause.

Transfer Challenge: ML Model Serving

A team building an ML model-serving service wants an assistant to write a data-validation function and its tests. Plan the task:

• the requirement and acceptance examples (synthetic records);
• the context to share and not to share;
• what the independent checks are;
• how you would check any suggested dependency;
• what the record would say about what the checks do not show about the model;
• what is different about AI in the product, such as the model itself, and why this chapter does not cover it.

Transfer Guidance (1 of 2)

Transfer Guidance (2 of 2)

Summary

• An assistant gives you candidates. Your job:
  • define the task;
  • keep the requirement and the examples yours;
  • check the output against evidence the assistant did not write;
  • record what you accepted and why.
• The failures are familiar: invented methods, APIs newer than the baseline, dependencies that do not exist, boundaries one step off, tests that repeat the code, abstractions nobody needs, policy changed in passing, claims of testing that nobody ran.
• Each has an inexpensive check. None of them requires a live service to practise.

Where to Go Next

• This is the last chapter of the book’s main sequence.
• The appendices that follow collect material the chapters point to: notation and a programming refresher, tools and reproducible setup, engineering templates, project versions and a glossary, and a guide to critical reading and project evidence.

Sources and Further Reading

1. GitHub Docs: Application card: GitHub Copilot inline suggestions (one vendor’s account of one tool).
2. Oracle: List (Java SE 21 and 17); javac (Java SE 17).
3. Apache Maven: maven-compiler-plugin: compile mojo (the release parameter).
4. OWASP: Top 10 for LLM Applications (2025 edition).
5. Spracklen et al.: We Have a Package for You!, arXiv 2406.10279.

• Full references and links are in the chapter text.