Expert Systems and Machine Vision

When a patient arrives with a fever and a rash, a good doctor does not search a graph; she applies hundreds of remembered rules of the form "if this and this, then probably that".

An expert system puts such rules in a program, separate from the engine that applies them, so the knowledge of a scarce human specialist can be copied, explained and used at any hour.

Machine vision gives computers eyes: it turns a grid of pixel numbers into edges, objects and decisions. Both are classic applications that combine the ideas of the earlier units.

6.1 Expert systems

An expert system (ES) is a program that solves problems in a narrow domain at the level of a human expert, using a body of expert knowledge held in an explicit knowledge base.

Famous examples: DENDRAL (1960s, infers molecular structure from mass-spectrometer data), MYCIN (1970s, diagnoses blood infections and recommends antibiotics from about 600 rules; its evaluation found its advice comparable with specialists', though it was never used clinically), XCON/R1 (configures computer orders for DEC, with thousands of rules, saving the company a great deal of money) and PROSPECTOR (mineral exploration).

Modern descendants are business-rule engines, tax and loan-eligibility advisers and clinical decision-support tools.

Characteristics

Expert systems are domain-specific, knowledge-based (knowledge separated from the reasoning engine), able to explain their conclusions (why are you asking this?

how did you reach that?), able to reason with uncertain and incomplete information, and modifiable by editing rules without rewriting the program.

The people involved are the domain expert (who has the knowledge), the knowledge engineer (who encodes it) and the user. Expert system shells (CLIPS, Jess, Drools) provide the engine, interface and explanation, leaving only the knowledge base to be written.

Inference: forward and backward chaining

Forward chaining (Unit 3) suits monitoring and diagnosis from symptoms: start from the observed facts and derive what follows.

Backward chaining suits hypothesis testing: start from a goal ("is it flu?") and ask only the questions that matter.

Conflict resolution picks one rule when several can fire: highest priority, most specific, most recent facts, or first listed.

Reasoning with uncertainty: certainty factors

Expert knowledge is rarely certain. MYCIN used certainty factors (CF) between −1 (certainly false) and +1 (certainly true), 0 meaning no evidence.

• A rule IF A AND B THEN C with rule certainty CF_r gives CF(C) = CF_r × min(CF(A), CF(B)) (for OR use max).
• Two rules supporting the same conclusion with CF₁ and CF₂ (both positive) combine as CF = CF₁ + CF₂ (1 − CF₁); for CF₁ = 0.6 and CF₂ = 0.5, CF = 0.6 + 0.5 × 0.4 = 0.8: independent evidence reinforces, but never exceeds 1.
• Evidence of opposite signs combines as (CF₁ + CF₂) / (1 − min(|CF₁|, |CF₂|)); for 0.6 and −0.4 this gives 0.2 / 0.6 = 0.333.

For premises with CF 0.8 and 0.6 and a rule with CF_r = 0.9: CF = 0.9 × min(0.8, 0.6) = 0.54. Certainty factors are heuristic and not true probabilities; Bayesian networks (Unit 3) are the principled alternative.

Stages of expert-system development

1. Identification: define the problem, the experts, the resources and the goals; decide whether an ES is suitable (a narrow, well-understood domain with a real expert who is available).
2. Conceptualisation: find the key concepts, relations and the problem-solving strategy.
3. Formalisation: choose a representation (rules, frames) and a tool or shell.
4. Implementation: encode the knowledge base and the interface; build a prototype quickly and grow it.
5. Testing and evaluation: compare against expert judgement and real cases, refine rules, check coverage and conflicts.
6. Maintenance: update rules as the domain changes; an ES whose knowledge ages is dangerous.

Advantages and limitations

Advantages: consistent decisions, availability, preservation of rare expertise, explanations, training value.

Limitations: the knowledge-acquisition bottleneck (experts find it hard to state what they know), brittleness at the edge of the domain (the system does not know what it does not know), no common sense, no learning unless added, and maintenance cost as the rule base grows into thousands of interacting rules.

These are the reasons expert systems were overtaken by machine learning for perception-like tasks, though rules remain strong where transparency and legal accountability matter.

6.2 Machine vision

Computer or machine vision extracts information from images and video so that a machine can act on it.

A digital image is a grid of pixels, each a number: 0–255 for an 8-bit grey level (0 black, 255 white), or three numbers for red, green and blue.

A 1920 × 1080 colour image has 2,073,600 pixels and 6,220,800 numbers (about 6 MB uncompressed).

Steps of machine vision

1. Image acquisition: a camera or scanner captures the scene; lighting, resolution and lens matter as much as software.
2. Preprocessing: improve the image: noise removal (smoothing filters, the median filter), contrast enhancement (histogram equalisation), resizing, conversion to grey.
3. Segmentation: divide the image into regions or objects: thresholding (pixel > T becomes foreground), edge detection, region growing, clustering.
4. Feature extraction: measure properties of each region: area, perimeter, shape, colour histogram, texture, corners.
5. Classification and recognition: decide what each object is, using rules or a learned classifier (k-NN, a neural network; convolutional neural networks learn the features themselves).
6. Interpretation and action: understand the scene (a count, a defect, a name, a position) and act: reject a part, open a gate, report a result.

Vision is described at three levels: low-level (pixels in, pixels out: filtering, edges), mid-level (regions, shapes, motion) and high-level (objects, scenes, meaning).

Edge detection by convolution

An edge is a place where brightness changes quickly. A filter (kernel) is a small matrix slid over the image; at each position the products of kernel and pixels are summed (convolution). The Sobel kernels respond to horizontal and vertical brightness changes:

Gx = [[−1, 0, 1], [−2, 0, 2], [−1, 0, 1]] and Gy = [[−1, −2, −1], [0, 0, 0], [1, 2, 1]].

The edge strength is √(Gx² + Gy²).

Applications

Quality inspection in factories (finding defects at line speed), optical character recognition (including Devanagari scripts and number plates), face detection and recognition, medical imaging (spotting tumours or retinal disease), traffic monitoring, agriculture (plant disease from leaf photos, crop counting from drone images), robot navigation and self-driving cars, document scanning and satellite image analysis (landslide and flood mapping, important for Nepal).

Concerns: privacy and surveillance, bias (face recognition accuracy differs across groups), and the need for human oversight of high-stakes decisions.

Challenges

Variation in lighting, viewpoint, scale and occlusion; the vast number of object classes; 3-D structure inferred from 2-D projection; the need for large labelled datasets; adversarial examples (tiny, deliberate pixel changes that fool a classifier).

Practising the commands

The first program is a small forward-chaining inference engine with certainty factors, running the rule base of Section 6.1.

rules = [("R1", {"fever", "cough"}, "flu_suspect"), ("R2", {"flu_suspect", "body_ache"}, "flu"),
         ("R3", {"flu"}, "advise_rest_and_fluids"), ("R4", {"rash", "fever"}, "measles_suspect")]
facts = {"fever", "cough", "body_ache"}; cycle = 0
while True:
    fired = [(n, c) for n, p, c in rules if p <= facts and c not in facts]
    if not fired: break
    cycle += 1; name, concl = fired[0]; facts.add(concl)
    print("cycle", cycle, "fire", name, "->", concl)
print("final facts:", sorted(facts), "| measles derived:", "measles_suspect" in facts)
combine = lambda a, b: a + b * (1 - a) if a >= 0 and b >= 0 else (a + b) / (1 - min(abs(a), abs(b)))
print("CF combine 0.6, 0.5 ->", round(combine(0.6, 0.5), 3), "| 0.6, -0.4 ->", round(combine(0.6, -0.4), 3))
print("rule CF 0.9 with premises 0.8, 0.6 ->", round(0.9 * min(0.8, 0.6), 2))

cycle 1 fire R1 -> flu_suspect
cycle 2 fire R2 -> flu
cycle 3 fire R3 -> advise_rest_and_fluids
final facts: ['advise_rest_and_fluids', 'body_ache', 'cough', 'fever', 'flu', 'flu_suspect'] | measles derived: False
CF combine 0.6, 0.5 -> 0.8 | 0.6, -0.4 -> 0.333
rule CF 0.9 with premises 0.8, 0.6 -> 0.54

The second program applies the Sobel filter to the 5 × 5 image of Section 6.2 and thresholds the result.

import math
image = [[0, 0, 10, 10, 10] for _ in range(5)]
GX = [[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]]; GY = [[-1, -2, -1], [0, 0, 0], [1, 2, 1]]
def conv(img, k, r, c): return sum(k[i][j] * img[r + i - 1][c + j - 1] for i in range(3) for j in range(3))
print("Gx:")
for r in range(1, 4): print([conv(image, GX, r, c) for c in range(1, 4)])
print("edge strength:")
for r in range(1, 4): print([round(math.hypot(conv(image, GX, r, c), conv(image, GY, r, c))) for c in range(1, 4)])
edges = [["#" if math.hypot(conv(image, GX, r, c), conv(image, GY, r, c)) > 20 else "." for c in range(1, 4)] for r in range(1, 4)]
print("thresholded (> 20):"); print("\n".join("".join(row) for row in edges))
print("pixels in a 1920x1080 colour image:", 1920 * 1080, "numbers:", 1920 * 1080 * 3)

Gx:
[40, 40, 0]
[40, 40, 0]
[40, 40, 0]
edge strength:
[40, 40, 0]
[40, 40, 0]
[40, 40, 0]
thresholded (> 20):
##.
##.
##.
pixels in a 1920x1080 colour image: 2073600 numbers: 6220800

6.3 Seeing the ideas: expert systems and machine vision

This section ties each idea of the unit to a picture and to a computed figure.

Plate 6.1 shows the people and machines behind the first expert systems, and Figures 6.1 to 6.6 run the architecture, forward and backward chaining, certainty factors, the development stages and the cost of a growing rule base.

Plates 6.2 and 6.3 show how images are formed and captured, and Figures 6.7 to 6.13 treat an image as numbers, then clean, segment and filter it, with a small convolutional network and a fair way to score the result.

Plates 6.4 and 6.5 show vision at work on satellite and medical images. Every number in a caption was printed by the figure code.

(a) Shortliffe: MYCIN

(b) VAX 11/780 that XCON configured

(c) Mass analyser (DENDRAL data)

(d) Central monitoring of patients

(a) Neuron that responds to one orientation

(b) Bayer sensor: one colour per photosite

(c) Screen pixels: red, green, blue

(a) Plate-reading cameras

(b) Fingerprint sensor

(c) Barcode scanner

(d) Kathmandu junction

(a) Kathmandu Valley, true colour

(b) Same valley, false colour

(c) Langtang Valley after the 2015 avalanche

(a) Chest radiograph

(b) Retina photograph

6.3.1 Eight uses of expert systems and machine vision in Nepal

1. Loan and credit rules in banks. A bank writes its lending policy as rules: if the income is above a limit and the loan is below a share of the collateral value, then the application passes to the next step. A rule engine applies these rules in the same way every time and can show which rule led to a refusal (Figure 6.1). Branch staff still decide the borderline cases, and the rules must be changed whenever Nepal Rastra Bank issues a new directive.
2. Customs and tax rules. Deciding the duty on an imported item means following many rules about its type, origin and value. A rule-based adviser at a customs office can walk an officer through the questions, like the backward chaining of Figure 6.3, and ask only the questions that matter. This gives similar answers at Birgunj and at Tatopani and leaves a written trail for appeals.
3. Health protocols for health workers. Child-illness guidelines used in Nepal's health posts are written as if-then charts: if there is fever and a stiff neck, send the child to the hospital at once. A phone program can hold such rules and ask the questions in order, much as MYCIN did for infections (Plate 6.1). It helps a health assistant who works alone, and it must say clearly when a case lies outside its rules.
4. Crop and livestock advice. A farmer describes the colour of the leaves, the weather and the crop stage, and a rule base suggests the likely problem and what to try first. The rules come from agricultural experts, and the certainty factors of Figure 6.4 can show when two symptoms strengthen a diagnosis. Local experts should check the rules for each region, because a rule that works in the Tarai may be wrong in the hills.
5. Traffic cameras and number plates. Cameras like those of Plate 6.3 read number plates and count vehicles at busy Kathmandu junctions. The software first finds the plate, then separates the characters and recognises each one, the steps of Figure 6.8. It can help to find a stolen vehicle or to record a red-light offence, but it must be tested in rain, dust and poor light, and the photographs must be stored safely.
6. Fingerprints and photographs for identity. The National ID, e-passports and the voters' list all use photographs and fingerprints. A fingerprint reader (Plate 6.3) turns the ridges into an image, finds their features and compares them with the stored record. A match that is not certain should lead to a second check by a person, because worn fingers of farmers and labourers often give poor images.
7. Satellite maps of floods and landslides. Images such as those of Plate 6.4 show where land has changed after an earthquake or a monsoon flood. A program separates water, snow, forest and built-up land by their pixel values, the thresholding of Figure 6.10, and compares images taken before and after. This helps agencies such as the Department of Hydrology and Meteorology and ICIMOD to plan relief when roads are cut and nobody can visit.
8. Reading medical images. Software can mark suspicious regions on a chest X-ray or an eye photograph (Plate 6.5) so that the doctor looks at those first. In a health camp with a single doctor this can save hours, especially for screening tuberculosis. Because the system can raise false alarms and miss cases, its precision and recall must be measured as in Figure 6.13, and the doctor makes the final decision.

Formula and fact sheet

• Expert system = knowledge base + inference engine + working memory + explanation + interface; knowledge is separate from the engine.
• Certainty factors: CF(rule conclusion) = CF_r × min(premises); same sign: CF₁ + CF₂(1 − CF₁); opposite signs: (CF₁ + CF₂)/(1 − min|CF|).
• Development: identification, conceptualisation, formalisation, implementation, testing, maintenance.
• Vision pipeline: acquisition → preprocessing → segmentation → feature extraction → recognition → interpretation.
• Sobel: Gx, Gy kernels; edge strength √(Gx² + Gy²).

Chapter summary

• An expert system captures a specialist's rules in a knowledge base used by a general inference engine, explains its reasoning and handles uncertainty with certainty factors; it is brittle outside its domain and costly to maintain.
• Development goes from identification through formalisation to testing and maintenance, with the knowledge engineer bridging expert and program.
• Machine vision turns pixels into decisions by acquisition, preprocessing, segmentation, feature extraction and recognition; edge detection is a convolution with kernels such as Sobel's.
• Both technologies serve real applications in industry, health and agriculture, with privacy, bias and accountability concerns.

Practice

1. Define an expert system and list its main components.
2. Distinguish the roles of the domain expert, knowledge engineer and user.
3. Explain forward and backward chaining; run the rule base of Section 6.1 forward with the facts fever, cough, body_ache.
4. Combine certainty factors 0.6 and 0.5; and compute the conclusion CF for a rule with CF 0.9 whose premises have CF 0.8 and 0.6.
5. List the stages of expert-system development.
6. Give three advantages and three limitations of expert systems.
7. List the steps of machine vision with one operation for each.
8. What is a convolution kernel? Compute Gx for the vertical-edge example.
9. Name four applications of machine vision relevant to Nepal.
10. How many numbers are needed to store a 1920 × 1080 colour image?

Answers and worked solutions

1. A program that solves problems in a narrow domain at expert level using an explicit knowledge base. Components: knowledge base, working memory, inference engine, user interface, explanation facility, knowledge acquisition module.

2. The domain expert supplies the knowledge, the knowledge engineer elicits and encodes it, the user poses cases and consumes the answers.

3. Forward: start from facts and fire rules until nothing new appears (data-driven). Backward: start from a goal and prove premises as subgoals (goal-driven). Forward run: R1 → flu_suspect; R2 → flu; R3 → advise_rest_and_fluids; R4 never fires.

4. 0.6 + 0.5 × (1 − 0.6) = 0.8. Rule: 0.9 × min(0.8, 0.6) = 0.54.

5. Identification, conceptualisation, formalisation, implementation, testing and evaluation, maintenance.

6. Advantages: consistency, availability, preservation of expertise, explanation. Limitations: knowledge-acquisition bottleneck, brittleness at the domain edge, no common sense or learning, maintenance cost.

7. Acquisition (capture an image), preprocessing (denoise, equalise), segmentation (threshold, edges), feature extraction (area, shape), recognition (classify), interpretation and action.

8. A small matrix slid over the image, summing kernel × pixel products. For a pixel with left neighbours 0 and right neighbours 10, Gx = (10 − 0)(1 + 2 + 1) = 40.

9. Devanagari OCR for documents and number plates; leaf-disease detection from farmers' phone photos; landslide and flood mapping from satellite images; quality inspection in small factories.

10. 1920 × 1080 × 3 = 6,220,800 numbers (2,073,600 pixels).

Model answers for Unit 6

Q1. What is an expert system? Describe its architecture and characteristics, and compare it with a conventional program. (10 marks)

Introduction. An expert system is a program that solves problems in a narrow field at the level of a human specialist, using knowledge that is written down in an explicit knowledge base.

Well-known examples are DENDRAL (molecular structure from mass spectra), MYCIN (blood infections, about 600 rules) and XCON, which configured DEC computer orders (Plate 6.1).

Architecture (Fig. 6.1). The knowledge base holds the facts and the IF-THEN rules of the field. Working memory holds the facts of the case being solved.

The inference engine matches rules against the facts and decides which rule fires, by forward or backward chaining. The user interface asks questions and shows advice.

The explanation facility answers why a question was asked and how a conclusion was reached. The knowledge acquisition module helps the knowledge engineer to enter the rules that the domain expert supplies.

Characteristics. (1) It is limited to one domain. (2) Knowledge is kept apart from the engine, so rules can be edited without rewriting the program. (3) It explains its reasoning.

(4) It can work with uncertain data through certainty factors. (5) It gives the same answer to the same case at any hour.

Point	Conventional program	Expert system
Knowledge	mixed into the code	separate knowledge base
Method	fixed algorithm	search and inference over rules
Data	needs complete input	can work with partial or uncertain input
Explanation	none	shows the chain of rules
Change	edit and recompile	add or edit a rule

Conclusion. An expert system copies a scarce specialist's rules into a general engine. It is strong where decisions follow clear rules and must be explained, and weak outside its domain.

Q2. Distinguish forward chaining from backward chaining with an example. (5 marks)

In forward chaining (data-driven) the engine starts from the known facts and fires every rule whose conditions are true, adding new facts until nothing new appears.

In backward chaining (goal-driven) it starts from a goal and tries to prove it by proving the conditions of a rule that concludes it, asking the user only when a fact is unknown.

Example (Figs. 6.2 and 6.3). The rules are R1 IF fever AND cough THEN flu_suspect; R2 IF flu_suspect AND body_ache THEN flu; R3 IF flu THEN advise_rest; R4 IF rash AND fever THEN measles_suspect.

With the facts fever, cough and body_ache, forward chaining fires R1, R2 and R3 in three cycles and ends with 6 facts; R4 never fires. With rash added, 4 rules fire and the run ends with 8 facts.

Backward chaining with the goal flu asks about fever, cough and body_ache, 3 questions; if the patient has no fever it stops after 1 question.

Point	Forward	Backward
Start	facts	goal
Best for	monitoring, diagnosis from symptoms	testing one hypothesis
Questions	may collect many facts first	only those needed

Q3. What are certainty factors? Show how they are combined. (5 marks)

A certainty factor (CF) is a number from −1 (certainly false) through 0 (no evidence) to +1 (certainly true), used in MYCIN to describe how strongly the evidence supports a conclusion when knowledge is uncertain.

Each rule also has its own CF, which says how much the rule is trusted.

Three combination rules (Fig. 6.4).

1. One rule with several conditions: CF(conclusion) = CF of the rule × the minimum CF of the conditions (the maximum for OR). With a rule CF of 0.9 and conditions 0.8 and 0.6 we get 0.9 × 0.6 = 0.54.
2. Two rules, same sign: CF = CF₁ + CF₂ (1 − CF₁). With 0.6 and 0.5 this gives 0.6 + 0.5 × 0.4 = 0.80. Independent evidence strengthens the belief but the result never passes 1.
3. Opposite signs: CF = (CF₁ + CF₂) / (1 − min(|CF₁|, |CF₂|)). With 0.6 and −0.4 this gives 0.2 / 0.6 = 0.333.

Remark. Certainty factors are heuristics, not true probabilities; Bayesian networks are the principled alternative.

Q4. Explain the stages of expert-system development. State the advantages and limitations of expert systems. (10 marks)

Introduction. Building an expert system is a team effort of the domain expert, the knowledge engineer and the user. It follows six stages, and the work is repeated until the system agrees with the expert (Fig. 6.5).

1. Identification: define the problem, the expert, the resources and the goals, and decide that an expert system suits the task: a narrow field with a real, available expert.
2. Conceptualisation: find the key concepts, their relations and the problem-solving strategy.
3. Formalisation: choose a representation (rules or frames) and a tool or shell such as CLIPS or Drools.
4. Implementation: write the knowledge base and interface, and build a quick prototype.
5. Testing and evaluation: compare the answers with the expert's and with real cases, and correct the rules.
6. Maintenance: update the rules when the field changes; stale knowledge is dangerous.

Advantages. Consistent decisions; available at any hour; rare expertise is preserved; reasoning can be explained; useful for training.

Limitations. The knowledge-acquisition bottleneck: experts find it hard to say what they know. Brittleness: the system does not know when a case lies outside its field. No common sense and no learning unless added.

Maintenance cost: for n rules there are n(n−1)/2 pairs that may clash, which is 179,700 for 600 rules and 1,999,000 for 2,000 (Fig. 6.6).

Aspect	Advantage	Limitation
Knowledge	kept and shared	hard to collect
Decisions	consistent	fail at the edge of the domain
Change	edit rules	large rule bases clash

Conclusion. Expert systems remain useful where rules are clear and decisions must be explained, such as loan policy and health protocols, while learning methods have taken over perception tasks.

Q5. Describe the steps of machine vision with a small example. (10 marks)

Introduction. Machine vision extracts information from images so that a machine can act.

A digital image is a grid of pixels, each a number (Fig. 6.7): a grey pixel is 0 to 255, and a colour pixel has three numbers for red, green and blue.

A full-HD colour image has 2,073,600 pixels and 6,220,800 numbers, about 5.93 MiB uncompressed.

Steps (Fig. 6.8).

1. Image acquisition: a camera or scanner captures the scene; lighting and lens matter as much as software (Plate 6.3).
2. Preprocessing: remove noise (median filter) and improve contrast. In the example a 3 × 3 median filter removes five bad pixels.
3. Segmentation: split the image into regions by thresholding, edges or region growing. Thresholding at 120 turns the 12 × 12 image into object and background.
4. Feature extraction: measure each region: area, perimeter, shape, colour. The example finds two regions of 12 and 16 pixels.
5. Classification and recognition: decide what each object is with rules, k-NN or a convolutional network.
6. Interpretation and action: use the result: count, reject a part, open a gate.

Level	Input to output	Examples
Low	pixels to pixels	filtering, edges
Middle	pixels to regions	segmentation, shape, motion
High	regions to meaning	object recognition, scene understanding

Without the median filter, thresholding of the raw example gave 5 regions (three single-pixel specks and two objects with holes), so the later steps would be wrong.

Conclusion. Each step prepares the next, so an error early in the chain, such as noise, spreads to the final decision.

Q6. What is a convolution kernel? Detect a vertical edge with the Sobel filter. (5 marks)

An edge is a place where brightness changes quickly. A kernel is a small matrix that is slid over the image; at each position the products of the kernel and the pixels under it are added, which is called convolution.

The Sobel kernels are Gx = [[−1, 0, 1], [−2, 0, 2], [−1, 0, 1]] for changes from left to right and Gy = [[−1, −2, −1], [0, 0, 0], [1, 2, 1]] for changes from top to bottom. The edge strength is √(Gx² + Gy²).

Example (Fig. 6.9). Take a 5 × 5 image in which every row is 0, 0, 10, 10, 10. For the pixel in column 1 the left column holds 0 and the right column holds 10, so Gx = 10 × (1 + 2 + 1) − 0 = 40.

In column 2 the left column is 0 and the right is 10, so Gx = 40. In column 3 both sides are 10 so Gx = 0. Every column is constant from top to bottom, so Gy = 0.

The result [40, 40, 0] marks the edge, two pixels wide because the 3 × 3 window sees it from both sides.

On a bright 4 × 4 square on a dark background the same filters give 40 along the sides, up to 42.4 at the corner pixels and 0 in flat areas, so the outline of the square appears.

Q7. Explain noise removal and thresholding in preprocessing and segmentation. (5 marks)

Noise removal. Sensors add random errors such as salt (white) and pepper (black) pixels. A mean filter replaces a pixel by the average of its neighbours, which reduces noise but also blurs edges.

A median filter replaces it by the middle value of the sorted neighbours, which ignores one extreme value.

Example (Fig. 6.11). On a scan line with a spike of 200 among values of 10, the 3-point mean gives 73.3 on three pixels, while the median gives 10.

At a step from 10 to 60 the mean produces the blurred values 26.7 and 43.3, but the median keeps 10 and 60, so the edge stays sharp. A dropout of 0 among 60s becomes 40.0 under the mean and 60 under the median.

Thresholding. A pixel above a threshold T becomes object, the others background. Otsu's method tries every level and keeps the one that separates the two groups best.

In Fig. 6.10, for 2,000 pixels, it chooses T = 124; the groups overlap, so 110 pixels (5.5 percent) are still on the wrong side.

Use. Filter first, then threshold, because noise would otherwise create false regions.

Q8. Discuss applications and challenges of machine vision, and say how a convolutional network helps. (10 marks)

Introduction. Machine vision turns pixels into decisions. Cameras, scanners and satellites supply the images (Plates 6.3 and 6.4), and software finds edges, regions and objects.

Applications. (1) Factory inspection finds cracks and missing parts at line speed. (2) Character recognition reads documents and number plates. (3) Biometrics match fingerprints and faces.

(4) Medical imaging marks suspicious regions on X-rays and retina photographs (Plate 6.5). (5) Traffic monitoring counts and classifies vehicles. (6) Agriculture detects leaf disease from phone photos.

(7) Satellite mapping finds floods, snow and landslides, such as the Langtang avalanche of 2015. (8) Robots and vehicles navigate by seeing.

Challenges. Changes of light, viewpoint and scale; objects hidden by others; many object classes; recovering 3-D shape from a flat image; the need for large labelled sets; adversarial images with tiny changes that fool a classifier; and privacy and bias, because face systems can be less accurate for some groups.

Convolutional networks (Fig. 6.12). A convolutional network learns its own filters.

In the figure a 28 × 28 image goes through a 3 × 3 convolution with 8 filters (26 × 26 × 8, 80 parameters), pooling (13 × 13 × 8), a second convolution with 16 filters (11 × 11 × 16, 1,168 parameters), pooling (5 × 5 × 16 = 400) and a dense layer of 10 outputs (4,010 parameters): 5,258 parameters in all.

A dense network 784-128-10 needs 101,770, which is 19.4 times more, because a filter reuses the same 9 weights at every position.

Method	Features	Strength	Weakness
Classical (edges, thresholds)	designed by hand	fast, needs little data	fails when conditions change
Convolutional network	learned from data	robust, accurate	needs many labelled images

Conclusion. Vision succeeds when images are controlled and tested, and humans must stay in charge of high-stakes decisions.

Q9. A vision system inspects 1,000 bottles, of which 50 are cracked. It finds 45 and raises 13 false alarms. Evaluate it. (5 marks)

Confusion matrix (Fig. 6.13). True positives TP = 45 (cracked and flagged). False negatives FN = 50 − 45 = 5 (cracked and missed). False positives FP = 13 (good but flagged). True negatives TN = 1000 − 45 − 5 − 13 = 937.

Measures.

• Precision = TP / (TP + FP) = 45 / 58 = 0.7759: of the bottles flagged, 77.6 percent are really cracked.
• Recall = TP / (TP + FN) = 45 / 50 = 0.90: the system finds 90 percent of the cracked bottles.
• F1 = 2PR / (P + R) = 0.8333, a single score that balances the two.
• Accuracy = (45 + 937) / 1000 = 0.982.

Why accuracy alone misleads. A system that never flags a bottle has accuracy 950 / 1000 = 0.950 but recall 0, because it finds no cracked bottle. When defects are rare, report precision and recall.

Judgement. The system is good: it catches 9 of 10 defects, and 13 false alarms per 1,000 bottles cost only a second look. If a cracked bottle is dangerous, the threshold should be changed to raise recall even if more false alarms appear.