23ECE467 Quiz 2 Masterguide - Channels, Fields, Polynomials, Cyclic Codes

1 · Exam Intel — what Quiz 2 actually is

Syllabus told by sir: Channel models / capacity, polynomials, cyclic codes (systematic + non-systematic), groups and fields. When class asked “only ppts 2, 3, 8?”, he said yes.

Topic	Read this
Channel models + capacity	`Information_Theory_and_Coding (1).pdf` slides 49–80 (Unit-II)
Groups + Fields + Polynomials	`LECTURE SLIDE 2.pdf` (all 42 slides)
Extended GF(2^m), minimal poly, vector space	`LECTURE SLIDE 3.pdf` (14 slides)
Systematic vs non-systematic idea + trellis/Viterbi	`LECTURE SLIDE 8.pdf` (convolutional example)

Gap you must know: none of the 3 PPTs has a full cyclic-code encoder page. Slide 8 is convolutional, not cyclic. But cyclic needs no new math: it is just polynomial division over GF(2) (Slides 2+3) + systematic idea (Slide 8). Prof expects you to combine them: c=m·g vs c=x^{n-k}m + rem.

Course Plan mapping: Lectures 9–15 = CO3 channels, Lectures 16–25 = CO4 Galois/poly/linear block, Lectures 26–40 = cyclic + convolutional + Viterbi. Quiz 2 sits at the 9–25 boundary, hence this mix.

2 · Formula Sheet (memorize)

Channels

Row sums to 1: Σ_j P(y_j|x_i)=1

P(x,y)=P(y|x)P(x) · Bayes: P(x|y)=P(y|x)P(x)/P(y)

P(Y)=P(X)·M (row vector × matrix) · H(Y|X)=Σ P(x_i)·H(row i)

BSC C = 1 − H_b(ε), max at P(0)=P(1)=½

BEC C = 1 − γ

C = max_P(X) I(X;Y) · I=H(Y)−H(Y|X)

R = log₂M / n · Noisy theorem: H/T_s ≤ C/T_c ⇒ reliable

Shannon–Hartley: C = B log₂(1+S/N) · S_lin=10^dB/10

Fields + Polynomials + Cyclic

GF(2): 1+1=0 (XOR add, AND mul) · GF(p): mod-p ops

a^q−1=1 · order divides q−1 · primitive generates all non-zero

f = q·g + r, deg r < deg g · f(1)=0 ⇒ (X+1) divides f

Irreducible: no divisor · Primitive deg m: smallest n with p|Xⁿ+1 is n=2^m−1

Non-systematic cyclic: c(x)=m(x)·g(x)

Systematic cyclic: c(x)=x^n−km(x)+r(x), r=rem(x^n−km / g)

Check: c mod g = 0 · h=(xⁿ+1)/g · syndrome s=r mod g

3 · Watch ONLY this — your playlist, in order

Playlist PLXnsjPD8-xutVH9OHMzeBHFc-PYaEZ6AV (Vaishali). Same teacher, same notation. Durations from yt-dlp. Skip everything else if time is short.

#	Video	Time	Why + slide match
1	L5 DMC	14:11	Channel matrix. Main deck 49-54.
2	L6 Hartley	16:39	Capacity gap filler. Slide 80.
3	L9 Linear Block	11:13	GF(2) background for cyclic. Slide 2.
4	L10 Block Question	9:35	Practice. Same.
5	L12 Cyclic Intro ★	16:54	Start cyclic here.
6	L13 Poly Sys/Non-sys ★★	12:05	Prof hint. Watch twice. Do Q14 with it.
7	L14 Matrix	8:52	G/H method.
8	L15 Decoding	17:45	Syndrome.
9	L16 Conv Intro	12:42	Slide 8 start.
10	L17 Tree	9:17	Slide 8 tree.
11	L19 State	8:01	Slide 8 FSM.
12	L20 Trellis	6:51	Slide 8 trellis.
13	L18 Viterbi	7:11	Slide 8 decode.

Total 2h31m, 1.5x ≈ 1h41m. No time? 58 min: L13 (12:05) + L15 (17:45) + L5 (14:11) + L20 (6:51) + L18 (7:11). Skip L1-L4 (Unit-I), L21 Sequential, L22 Turbo — not Quiz 2. Slides 2+3 proofs (mod-5 tables, Lagrange, GF(7), GF(4) α²=1+α, conjugates) are NOT in playlist — read slides for those.

4 · MIT OpenCourseWare

MIT 6.441 Information Theory — BSC/BEC capacity, Ch.4–5 MIT 6.02 Digital Comm — noisy channels, Shannon-Hartley, Viterbi MIT 6.450 Digital Comm I (Gallager) — GF(2), polynomials, cyclic MIT 6.451 Digital Comm II — systematic vs non-systematic, convolutional

Also: Prof. Ranjan Bose NPTEL course 117102062 (IIT Delhi) on Ansys/NPTEL portal covers linear block + Galois + cyclic + division algorithm — same as textbook T1. Duke ECE590 cyclic slides (pfister.ee.duke.edu) give the cleanest systematic proof: p = −x^n−km mod g.

5 · Books from Course Plan — where to open

Book	Open this for Quiz 2
Bose 2nd ed	Ch.2.2–2.7 channels + BSC/BEC + Hartley; Ch.4.2–4.4 poly division + cyclic `c=m·g` vs remainder-append; Ch.5.2–5.4 GF/minimal poly; Ch.6 trellis/Viterbi
Satyanarayana	Ch.3 DMC matrix, Ch.4 Gaussian/Hartley, Ch.6 block-code algebra + GF tables, Ch.7 cyclic systematic + syndrome, Ch.8 convolutional
Lin & Costello 2nd ed	Ch.1 BSC/AWGN models; Ch.2 groups/rings/fields + GF(2^m) + minimal poly; Ch.5 cyclic encode (5.4–5.6) + syndrome; Ch.11–12 convolutional + Viterbi
Wells	Ch.2 DMC + symmetric `C=log\|Y\|−H(row)` + BSC/BEC; Ch.4–5 binary fields + poly mod `xⁿ+1` + systematic encoder + Meggitt; Ch.6 trellis/Viterbi/puncturing

If short on time: Bose Ch.2 + Ch.4.4 + Lin Ch.2.5–2.7 + Ch.5.4 covers 90% of Quiz 2.

6 · Channels in plain words

Think of a channel as a noisy pipe: you put a symbol x in, you get a (possibly different) symbol y out. A Discrete Memoryless Channel (DMC) is described by one table — the channel matrix M where M[i][j] = P(y_j|x_i).

Each row sums to 1: Σ_j P(y_j|x_i) = 1 — "if I send this, something must come out."

Recipe you always follow: P(Y) = P(X) · M (row vector × matrix) → joint = scale row i by P(x_i) → H(Y|X) = weighted average of row entropies → I = H(Y) − H(Y|X) → capacity C = max_P(X) I.

Layman check: H(Y|X) = "how confused am I about the output even after I know what was sent?" If rows are sharp (e.g. [1,0]), confusion is 0. If rows are coins ([½,½]), confusion is 1 bit.

Binary Symmetric Channel (BSC) — flips with prob ε

Input {0,1}, output {0,1}. Each bit is flipped with probability ε, passed correctly with 1−ε. Matrix: [[1−ε, ε],[ε, 1−ε]]. Symmetric → capacity is reached at uniform input P(0)=P(1)=½.

BSC capacity: C = 1 − H_b(ε), at P(0)=P(1)=½. H_b = binary entropy.

Example: ε=0.1 → H_b ≈ 0.469 → C ≈ 0.531 bits/use. ε=0.25 → C ≈ 0.189. ε=0.5 → C=0 (output is a coin toss, tells you nothing).

Binary Erasure Channel (BEC) — erases with prob γ, never flips

Input {0,1}, output {0,e,1} where e = "lost". A bit is erased with probability γ and received perfectly otherwise. The receiver knows which positions were erased — that is why BEC hurts less than BSC.

BEC capacity: C = 1 − γ (no entropy math needed — the fraction that survives).

Example: γ=0.2 → C=0.8 bits/use. Compare: a 10% flip (BSC ε=0.1) gives only 0.53, but a 20% erasure still gives 0.8 — erasures are cheaper because you know where they happened.

Capacity curve C = 1 − H_b(ε) — what the shape tells you

The BSC curve is a U-shape: full capacity 1 at both ends (ε=0 perfect wire, ε=1 perfectly inverted wire — just flip every bit back!), zero at the middle (ε=0.5, pure coin toss). In exams only 0 ≤ ε ≤ 0.5 matters; beyond 0.5 you would relabel 0/1 and be back on the left half.

Peaking notes (say this in the exam): C=1 at ε=0 (no noise) and at ε=1 (deterministic flip — still 1 bit/use after inversion). Minimum C=0 at ε=0.5 (output independent of input). Steep near the edges, flat near 0.5 — so a small ε like 0.1 already costs ~0.47 bits.

Shannon–Hartley worked example: B = 4 kHz, SNR = 10 dB

C = B · log₂(1 + S/N), S_lin = 10^dB/10

Convert dB → linear (the step that usually goes wrong): S_lin = 10^10/10 = 10¹ = 10. So S/N = 10 (not 10!).
Add 1 inside the log: 1 + S/N = 11. (Forgetting the +1 is the classic 0-mark mistake.)
Base-2 log via natural logs: log₂(11) = ln(11)/ln(2) ≈ 2.3979/0.6931 ≈ 3.459.
Multiply by bandwidth: C = 4000 × 3.459 ≈ 13,837 bps ≈ 13.8 kbps.

Answer: C ≈ 13.8 kbps (B=4 kHz, SNR=10 dB → S/N=10 → 4000·log₂11)

Exam tricks: 4.77 dB ≈ ratio 3 (→ log₂4=2, C=8 kbps); 30 dB = 1000 (→ 1001 ≈ 1024=2¹⁰, C ≈ 10 Mbps for B=1 MHz). If your S/N looks like the dB number itself, you forgot the 10^dB/10 conversion.

7 · Galois fields — the only 10 ideas you need

1. Group vs field in 1 line: group = add + undo. Field = add + undo and multiply + undo (except 0). Coding lives in fields so you can divide polynomials for parity.

2. GF(2) = {0,1}: add = XOR (1+1=0), mul = AND. This is 90% of Quiz 2. GF(3) = {0,1,2} mod-3. GF(p) only if p is prime.

GF(2) +	0	1	GF(2) ×	0	1
0	0	1	0	0	0
1	1	0	1	0	1

3. Characteristic is prime: smallest λ with 1+1+…+1 (λ times) = 0. GF(2): 1+1=0 → λ=2. GF(7): λ=7. Characteristic 2 is why subtraction = addition (so long division uses XOR, never borrow).

4. a^q−1=1: in GF(q), every non-zero to power q−1 = 1. Order of a divides q−1. Ex GF(7): 3→3,2,6,4,5,1 (order 6, primitive). 4→4,2,1 (order 3, not primitive). Exam trick: order must divide q−1, so in GF(7) only 1,2,3,6 are possible orders.

5. GF(4) is NOT 0,1,2,3 mod-4: 4 not prime, 2 has no inverse mod-4 (2×anything is even, never 1). So mod-4 is a ring, not a field. Build GF(4) as an extension: start from GF(2), adjoin a root α of the irreducible X²+X+1=0, i.e. impose α²=1+α (since −=+, X²=X+1 means α²=α+1). Elements: {0, 1, α, α²=1+α}. Powers cycle with length 3 = 2²−1:

GF(4) +	0	1	α	1+α	GF(4) ×	1	α	1+α
0	0	1	α	1+α	0	0	0	0
1	1	0	1+α	α	1	1	α	1+α
α	α	1+α	0	1	α	α	1+α	1
1+α	1+α	α	1	0	1+α	1+α	1	α

How to read: add in poly form (XOR coefficients: 1+α plus α = 1). Multiply in power form (add exponents mod 3: α·α²=α³=1). Every non-zero has an inverse: 1⁻¹=1, α⁻¹=α², (α²)⁻¹=α.

6. GF(2^m) recipe: take primitive degree-m poly p(X), set p(α)=0. Ex GF(16): p=X⁴+X+1 → α⁴=1+α. Then {0,1,α,…,α¹⁴}, α¹⁵=1. Poly form for add, power form for mul. Degree-m primitive poly ⇒ cycle length exactly 2^m−1 (maximal).

7. Conjugates: if β root, so are β²,β⁴,… (Frobenius: squaring permutes roots in char 2). Minimal poly = product (X+conjugates). Ex β=α³ in GF(16): conjugates α³,α⁶,α¹²,α⁹ → minimal X⁴+X³+X²+X+1. Minimal poly is always irreducible; degree divides m.

8. Why for cyclic? Codewords are multiples of g. GF guarantees division works, remainder degree < deg g always exists, so systematic parity r = rem(x^n−km/g) is unique. Without a field, q and r would not be unique and syndrome decoding would break.

Memorize: GF(2):1+1=0 · a^q−1=1 · GF(2^m) has 2^m elements, α^{2^m−1}=1

Drill: write GF(3) add/mul tables from memory. Find inverses: 1⁻¹=1, 2⁻¹=2. Solve 2x+1=0 → x=1 (see Q9). List powers of α in GF(4) until back to 1.

8 · Polynomials over GF(2) — add / mul / div / irreducible

Bits = coefficients: 1011 = X³+X+1. Add = XOR columns, no carries. Mul = distribute + XOR. Div: f = q·g + r, deg r < deg g.

Add ex: (X³+X+1)+(X²+X+1) → 1011⊕0111=1100 → X³+X² (X and 1 cancel).

Mul ex: (X²+1)(X+1)=X³+X²+X+1. Note (X+1)²=X²+1 (middle 2X=0 — Freshman's dream in char 2).

Div ex (Slide 2, must do by hand): (X⁶+X⁵+X⁴+X+1) ÷ (X³+X+1). Long division = repeat: match leading term, XOR, bring down. Subtraction IS addition here.

Why step 3 above looks odd: the dividend has a missing X³ and X² term (zeros), so after step 2 the remainder is X²+X+1 already below degree 3 — division stops. Full XOR chain: start 1110011 → after X³: 0101011 → after X²: 0000111 → deg 2 < 3, done. q=X³+X², r=111→X²+X+1. Always verify f=q·g+r.

X⁶÷X³=X³ → rem X⁵+X³+X+1 → +X² → rem X²+X+1 → stop. q=X³+X², r=X²+X+1 ✓

Root test: f(1)=0 ⇒ (X+1) divides f (since X=1 root ⟺ factor X+1 in char 2, where X−1=X+1). Ex X⁴+X³+X²+1 at X=1: 1+1+1+1=0 → divisible. Similarly f(0)=0 ⇒ X divides f (no constant term).

Irreducible vs primitive — which question are you answering?

Irreducible checklist: deg 2–3: check f(0)≠0 and f(1)≠0 → done (any factor would be linear). Deg ≥4: also rule out product of irreducible quadratics (or higher). Trap: X⁴+X²+1 = (X²+X+1)², no root but reducible — root test is necessary, not sufficient, above degree 3.

Primitive: irreducible deg m dividing Xⁿ+1 smallest at n=2^m−1. Ex X²+X+1 divides X³+1 → primitive, builds GF(4). X³+X+1: f(0)=1,f(1)=1 → irreducible → check divides X⁷+1 (and no smaller Xⁿ+1: n=1,7 divisors — check n=1,7; X+1 does not divide it) → primitive, builds GF(8).

For cyclic: need g | Xⁿ+1. (7,4) g=X³+X+1 divides X⁷+1 ✓. Then h=(X⁷+1)/g, syndrome s = r mod g.

Drill: divide (X⁵+X+1)/(X³+X+1) → q=X²+1, r=X² (Q12). Do Q10–Q12 twice without looking. Then verify every division with f=q·g+r.

9 · Cyclic codes — systematic vs non-systematic

prof hint

Non-systematic: c(x) = m(x)·g(x) — message hidden

Systematic: x^n−km(x) ÷ g(x) → remainder r(x) · c(x)=x^n−km(x)+r(x) — message visible + parity

(7,4) demo, g=X³+X+1, m=1101 (m=X³+X²+1):
Non-sys: m·g = X⁶+X⁵+X⁴+X³+X²+X+1 → 1111111.
Sys: X³m=X⁶+X⁵+X³ ÷ g → r=1 → c=X⁶+X⁵+X³+1 → 1101001 (message 1101 readable). Both satisfy c mod g=0.
Checklist: deg g=n−k, deg m<k, g | Xⁿ+1, h=(Xⁿ+1)/g, syndrome s=r mod g.

10 · Convolutional (Slide 8) in 4 lines

Block = combinational (no memory). Convolutional = sequential (memory m, 2^m states). Slide-8 code: rate ½, g0=1011, g1=1111, non-systematic feedforward. u=(10111) → v=(11,01,00,01,01,01,00,11). Poly form g0(D)=1+D²+D³. Viterbi = keep best path per state by Hamming distance (add-compare-select), traceback at end. Rate k/n, states 2^m.

9X · Cyclic + Convolutional — visual supplement

Verified graphs from Muse Spark explainers. Complements §§9–10 above.

9A · The one idea behind cyclic codes

A binary cyclic (n,k) code is the set of length-n multiples of one fixed generator polynomial g(x) of degree n−k, with the extra rule that g(x) divides xⁿ+1. That divisibility is what makes rotation of a codeword another codeword (cyclic property) and guarantees every remainder on division by g is unique and shorter than n−k bits.

Checklist before encoding: deg g = n−k · deg m < k · g | xⁿ+1 · check: c mod g = 0 · parity-check poly h = (xⁿ+1)/g

There are two ways to turn a message m(x) into a multiple of g — and this is exactly the systematic vs non-systematic distinction the professor flagged:

Non-systematic (multiply): c(x) = m(x)·g(x). Any product is trivially a multiple of g, so it is a valid codeword. Downside: the message bits are scrambled into the product — you cannot read m off c.
Systematic (shift → divide → append): shift the message left by n−k (x^n−km(x)), divide by g, keep remainder r(x), and transmit c(x) = x^n−km(x) + r(x). Because r ≡ x^n−km (mod g), the two cancel: c mod g = 0. The first k bits are the message verbatim, the last n−k bits are parity.

Memory hook: non-systematic = bake the message into the product (hidden). Systematic = park the message up front, staple the remainder behind it (visible + parity).

9B · Encoder block diagrams — message hidden vs visible

Non-systematic: message hidden

Systematic: message visible + parity appended

Why both are valid: the non-systematic product is divisible by g by construction; the systematic word is divisible because the appended remainder exactly cancels the remainder of the shifted message. Same code, different codewords for the same message.

9C · (7,4) worked example — m = 1101, g = X³+X+1

Message m = 1101 → m(x) = X³+X²+1, generator g = 1011 → X³+X+1 (deg 3 = n−k, and g | X⁷+1).

Path A — non-systematic: c = m·g

X³·g = X⁶+X⁴+X³ · X²·g = X⁵+X³+X² · 1·g = X³+X+1
XOR all (X³ occurs 3× → survives): c(x) = X⁶+X⁵+X⁴+X³+X²+X+1 → 1111111

Path B — systematic: shift, divide, append

X³m = X⁶+X⁵+X³ (1101000). Divide by g: X³·g → rem X⁵+X⁴ · X²·g → rem X⁴+X³+X² · X·g → rem X³+X · 1·g → rem 1 (parity 001)
c(x) = X⁶+X⁵+X³+1 → 1101001

Visual: same message, two codewords

Same m = 1101, same g	Non-systematic	Systematic
Formula	`c = m·g`	`c = X³m + rem(X³m/g)`
Codeword	`1111111`	`1101001`
Readable?	No	Yes — 1101 + 001
Check	`c mod g = 0` ✓	`c mod g = 0` ✓

10A · Convolutional codes — why a trellis exists

A block code is combinational (output depends only on the current input block). A convolutional code is sequential: the encoder holds the last m input bits in shift registers, so the output depends on the current input plus memory. With m memory bits there are 2^m states.

Slide-8 code: rate R = ½ (1 input → 2 outputs), generators g0 = 1011, g1 = 1111, memory m = 3 → 8 states, feedforward non-systematic. Input u = 10111 → v = (11,01,00,01,01,01,00,11).

Why a 4-state trellis below? Drawing all 8 states of the Slide-8 code is unreadable. The trellis below uses the classic textbook 4-state rate-½ code (K = 3, m = 2, g0 = 111, g1 = 101) to show the method: states as rows, time as columns, one solid branch per input-0, one dashed branch per input-1, each labelled input/output. The Slide-8 code works identically with 8 rows instead of 4.

10B · 4-state trellis with branch labels (m = 2)

Convention: state = (s1,s0) = last two inputs. Next = (u,s1). Outputs: c0 = u⊕s1⊕s0 (g0=111), c1 = u⊕s0 (g1=101). Solid = input 0, dashed = input 1. Label = u / c0c1.

State (s1,s0)	u=0 → next / out	u=1 → next / out
00	00 / 00	10 / 11
01	00 / 11	10 / 00
10	01 / 10	11 / 01
11	01 / 01	11 / 10

Verify one row yourself: state 10 means s1=1,s0=0. u=0 → c0=0⊕1⊕0=1, c1=0⊕0=0 → 10, next=(0,1)=01. ✓

10C · Viterbi ACS step — add, compare, select

At each trellis stage the decoder receives a noisy pair y. For every branch it computes a branch metric = Hamming distance d(y, label), adds it to the incoming path metric, and where two paths merge into one state it compares the two sums and selects (keeps) the smaller as the new survivor.

ACS per state: newMetric = min(pathA + d(y,labelA), pathB + d(y,labelB)) — keep winner, discard loser

Worked cleanly with y = 00: upper branch label 00 → d=0 → 2+0=2; lower branch label 11 → d=2 → 4+2=6; compare 2 vs 6 → keep upper, kill lower. Ties → either may be kept (exam-accepted arbitrary-tie rule). After the last stage, traceback from the best final state reads off u.

Slide-8 Viterbi recipe: branch metric = Hamming(y,label) → ACS at every state → traceback from best final metric.

Note on Slide-8 scale: with m=3 the same ACS runs at all 8 states per stage; the diagram above shows one such merge (2 paths → 1 survivor) which is identical in the 8-state trellis.

11 · 15 Practice Qs with solutions (click to open)

Do in order. Clean numbers chosen so logs collapse (½,¼,⅛, 1023/1024). If intermediate looks ugly, re-read the question.

Q1 — Matrix → P(Y), joint, H, I (uniform input)

Given: X ∈ {0,1}, P_X = [P(x0), P(x1)] = [½, ½]. Channel matrix M (row i = P(Y|x_i)):
Row0 = [P(y0|x0), P(y1|x0)] = [¾, ¼] · Row1 = [¼, ¾]. Check rows sum to 1: ¾+¼=1 ✓.

Step 1 — P(Y) = P(X)·M (column-weighted sums). Zero-background rule: P(yj) = Σ_i P(xi)·M[i][j].

P(y0) = P(x0)·¾ + P(x1)·¼ = ½·¾ + ½·¼ = ⅜ + ⅛ = 4/8 = ½. (Decimals: 0.375+0.125=0.5.)
P(y1) = P(x0)·¼ + P(x1)·¾ = ½·¼ + ½·¾ = ⅛ + ⅜ = 4/8 = ½. (0.125+0.375=0.5.)
So P_Y = [½, ½]. Check: ½+½=1 ✓. Symmetric input + symmetric channel ⇒ uniform output.

Step 2 — Joint P(x,y) = P(y|x)·P(x): scale each ROW i by P(xi).

Row0 × ½: [½·¾, ½·¼] = [⅜, ⅛] = [0.375, 0.125].
Row1 × ½: [½·¼, ½·¾] = [⅛, ⅜] = [0.125, 0.375].
Check total = ⅜+⅛+⅛+⅜ = 1 ✓. Column sums give back P(Y): col0 = ⅜+⅛=½ ✓.

P(x,y)	y=0	y=1
x=0	⅜ (0.375)	⅛ (0.125)
x=1	⅛ (0.125)	⅜ (0.375)

Step 3 — Entropies. First evaluate Hb(¼) term-by-term (log₂):

log₂(¼) = −2 exactly (since 2⁻²=¼). First term: −¼·log₂(¼) = −¼·(−2) = 0.5.
log₂(¾) = log₂3 − log₂4 = 1.58496 − 2 = −0.41504. Second term: −¾·(−0.41504) = 0.75×0.41504 = 0.31128.
Hb(¼) = 0.5 + 0.31128 = 0.81128 ≈ 0.811 bits.
H(X) = Hb(½) = 1 bit (uniform binary). H(Y) = 1 bit (P_Y=[½,½]).
H(Y|X) = Σ_i P(xi)·H(row i) = ½·Hb(¼) + ½·Hb(¼) = ½(0.81128)+½(0.81128) = 0.40564+0.40564 = 0.811 bits.
H(X,Y) = H(X)+H(Y|X) = 1 + 0.81128 = 1.811 bits.

Step 4 — Mutual information: I(X;Y) = H(Y) − H(Y|X) = 1 − 0.81128 = 0.18872 ≈ 0.189 bits.

Q2 — Non-uniform input

Given: P_X = [¼, ¾]. M = [[1,0],[½,½]]. Row0 sums 1+0=1 ✓, row1 ½+½=1 ✓.

Step 1 — Joint first (scale rows):

Joint row0 = ¼·[1,0] = [¼·1, ¼·0] = [¼, 0] = [0.25, 0].
Joint row1 = ¾·[½,½] = [¾·½, ¾·½] = [⅜, ⅜] = [0.375, 0.375]. (¾×½ = 3/8.)

P(x,y)	y=0	y=1
x=0	¼	0
x=1	⅜	⅜

Step 2 — P(Y) = column sums of joint:

P(y0) = ¼ + ⅜ = 2/8 + 3/8 = ⅝ = 0.625.
P(y1) = 0 + ⅜ = ⅜ = 0.375. Check 0.625+0.375=1 ✓. So P_Y=[⅝,⅜].

Step 3 — H(X), term by term: −¼·log₂¼ − ¾·log₂¾.

−¼·(−2) = 0.5. log₂(¾)=1.58496−2=−0.41504; −¾·(−0.41504)=0.31128.
H(X) = 0.5+0.31128 = 0.811 bits (same numbers as Hb(¼)).

Step 4 — H(Y) = −⅝·log₂⅝ − ⅜·log₂⅜:

log₂(⅝) = log₂5 − log₂8 = 2.32193 − 3 = −0.67807. Term: −0.625·(−0.67807) = 0.42379.
log₂(⅜) = log₂3 − log₂8 = 1.58496 − 3 = −1.41504. Term: −0.375·(−1.41504) = 0.53064.
H(Y) = 0.42379+0.53064 = 0.95443 ≈ 0.954 bits.

Step 5 — H(Y|X) = Σ P(xi)·H(row i):

H(row0)=H([1,0]) = −1·log₂1 − 0·log₂0 = −1·0 − 0 = 0 bits (noiseless row, no uncertainty).
H(row1)=H([½,½]) = −½·(−1) − ½·(−1) = 0.5+0.5 = 1 bit (fair coin).
H(Y|X) = ¼·0 + ¾·1 = 0 + 0.75 = 0.75 bits.
H(X,Y) = H(X)+H(Y|X) = 0.81128+0.75 = 1.561 bits.

Step 6 — I: I = H(Y)−H(Y|X) = 0.95443−0.75 = 0.20443 ≈ 0.204 bits.

Q3 — BSC p = ¼

Given: BSC crossover ε=¼. Capacity C = max_P(X) I(X;Y) = 1 − Hb(ε), achieved at uniform P(0)=P(1)=½.

Step 1 — Hb(¼) with every log shown:

Term A: −ε·log₂ε = −¼·log₂(¼) = −¼·(−2) = 0.5 (because ¼=2⁻²).
log₂(1−ε)=log₂(¾)=log₂3−2=1.58496−2=−0.41504.
Term B: −(1−ε)·log₂(1−ε) = −¾·(−0.41504) = 0.31128.
Hb(¼) = 0.5+0.31128 = 0.81128 ≈ 0.811 bits.

Step 2 — Capacity: C = 1 − 0.81128 = 0.18872 ≈ 0.189 bits/use.

Q4 — BSC p = ⅛

Given: BSC ε=⅛. C = 1 − Hb(⅛), max at uniform input.

Step 1 — Hb(⅛) term by term:

log₂(⅛) = −3 exactly (2⁻³=⅛). Term A: −⅛·(−3) = 3/8 = 0.375.
log₂(⅞) = log₂7 − log₂8 = 2.80735 − 3 = −0.19265.
Term B: −⅞·(−0.19265) = 0.875×0.19265 = 0.16856.
Hb(⅛) = 0.375+0.16856 = 0.54356 ≈ 0.544 bits.

Step 2 — Capacity: C = 1 − 0.54356 = 0.45644 ≈ 0.456 bits/use.

Q5 — BEC α = ¼

Given: BEC erases each bit with prob γ=α=¼, never flips. C = 1−γ at uniform input.

C = 1 − γ = 1 − ¼ = ¾ = 0.75 bit/use.
Intuition: ¼ of positions are "?" (erased), ¾ survive perfectly ⇒ ¾ info bit per use.
For n=1000 uses: max info bits = n·C = 1000×¾ = 3000/4 = 750 bits. (750 survive on average; 250 erased.)

Answer: C = ¾ bit/use; 1000 uses → 750 info bits max.

Q6 — Hartley B=4kHz SNR=4.77dB

Formula: C = B·log₂(1+S/N), with S_lin = 10^(dB/10). B=4000 Hz, SNR_dB=4.77.

Step 1 — dB → linear (show division first):

Exponent = dB/10 = 4.77/10 = 0.477.
S_lin = 10^0.477. Recall log₁₀3 = 0.4771, so 10^0.477 ≈ 3 (2.999). This dB value was chosen to give exactly 3.

Step 2 — Plug into Hartley: 1+S/N = 1+3 = 4. C = 4000·log₂4.

Step 3 — log₂4 = 2 (since 2²=4). C = 4000×2 = 8000 bps = 8 kbps.

Q7 — Hartley B=1MHz SNR=30dB

Formula: C = B·log₂(1+S/N). B=10⁶ Hz, SNR=30 dB.

Step 1 — dB → linear: exponent = 30/10 = 3. S_lin = 10³ = 1000.

Step 2 — Plug in: 1+S/N = 1001. C = 10⁶·log₂(1001).

Step 3 — Evaluate log₂(1001). Clean trick: 1024=2¹⁰:

log₂(1024) = 10 exactly. 1001 is just under 1024 (1001/1024=0.9775).
Exact: log₂(1001) = ln(1001)/ln2 = 6.9088/0.69315 = 9.967 ≈ 9.97. (Or log₁₀(1001)/log₁₀2 = 3.00043/0.30103 = 9.967.)
C = 10⁶ × 9.967 = 9.967×10⁶ bps ≈ 10 Mbps.

Q8 — Is ({0,1},+₂) a group? Field?

Background: group needs (closure, identity, inverse, associativity). Field needs Abelian group under + AND non-zero elements a group under ×, plus distributivity. Here +₂ = XOR (0+0=0,0+1=1,1+0=1,1+1=0), × = AND.

Step 1 — ({0,1},+₂) group test, all 4 sums:

Closure: 0+0=0 ✓, 0+1=1 ✓, 1+0=1 ✓, 1+1=0 ✓ — all stay in {0,1}.
Identity: 0, because 0+0=0 and 1+0=1 ✓.
Inverses: 0⁻¹=0 (0+0=0), 1⁻¹=1 (1+1=0=identity) — every element has one ✓.
Associative: (a+b)+c=a+(b+c) holds for XOR (check 8 cases or trust XOR) ✓. Commutative: a+b=b+a ✓.

⇒ ({0,1},+₂) IS an Abelian group.

Step 2 — ({0,1},×) group test:

Closure: 0·0=0,0·1=0,1·1=1 ✓. Identity: 1 (0·1=0,1·1=1) ✓.
Inverse: need x with 0·x=1. But 0·0=0, 0·1=0 — never 1. So 0 has no inverse ✗.

⇒ ({0,1},×) is NOT a group (it is only a monoid).

Step 3 — Field verdict: remove 0: {1} under × is trivially a group (1·1=1, inverse of 1 is 1). × distributes: a·(b+c)=a·b+a·c (verify e.g. 1·(1+1)=1·0=0; 1·1+1·1=1+1=0 ✓). Hence GF(2)=({0,1},+₂,×) IS a field.

Q9 — GF(3) tables + solve 2x+1=0

Step 1 — Addition mod 3 (add then subtract 3 if ≥3):

+ | 0  1  2
--+---------
0 | 0  1  2
1 | 1  2  0   (1+2=3→0)
2 | 2  0  1   (2+2=4→1)

Step 2 — Multiplication mod 3:

× | 0  1  2
--+---------
0 | 0  0  0
1 | 0  1  2
2 | 0  2  1   (2·2=4→1)

Step 3 — Inverses (element × inverse = 1): 1·1=1 ⇒ 1⁻¹=1. 2·2=4≡1 ⇒ 2⁻¹=2. (0 has none.)

Step 4 — Solve 2x+1=0 mod 3, one move per line:

2x + 1 = 0.
Subtract 1: 2x = −1. Mod 3, −1 ≡ 3−1 = 2. So 2x ≡ 2.
Multiply both sides by 2⁻¹=2: x ≡ 2·2 = 4 ≡ 4−3 = 1 mod 3.
Check: 2(1)+1 = 3 ≡ 0 ✓.

Q10 — Add (x³+x+1)+(x²+x+1)

Step 1 — Write as 4-column coefficient vectors (x³ x² x¹ x⁰), pad short poly with 0:

  x³ x² x¹ x⁰
  -----------
   1  0  1  1    (x³+x+1 = 1011)
⊕  0  1  1  1    (x²+x+1 = 0111)
  -----------
   ?  ?  ?  ?

Step 2 — XOR each column (GF(2): 0⊕0=0, 0⊕1=1, 1⊕1=0, no carries):

x³: 1⊕0 = 1. x²: 0⊕1 = 1. x¹: 1⊕1 = 0 (x cancels). x⁰: 1⊕1 = 0 (1 cancels).
Result row: 1 1 0 0 = 1100 = x³+x².

Answer: (x³+x+1)+(x²+x+1) = x³+x² → 1011⊕0111 = 1100.

Q11 — Multiply (x²+1)(x+1)

All arithmetic over GF(2) (add = XOR, 1+1=0). A=x²+1 (101), B=x+1 (11).

Step 1 — Distribute (FOIL): (x²+1)(x+1) = x²·x + x²·1 + 1·x + 1·1 = x³ + x² + x + 1.

Step 2 — Combine like terms (XOR columns x³ x² x¹ x⁰):

  x³ x² x¹ x⁰
   1  1  0  0    (x³+x² from x²·(x+1))
⊕  0  0  1  1    (x+1 from 1·(x+1))
   -----------
   1  1  1  1

No column has two 1s, so nothing cancels. Sum = x³+x²+x+1 = 1111.

Answer: (x²+1)(x+1) = x³+x²+x+1 (1111) ✓.

Note: (X+1)² = X²+2X+1 = X²+1 since 2X=0 in GF(2) — middle term vanishes, same rule.

Q12 — Divide (x⁵+x+1)/(x³+x+1) + irreducible?

Setup: f=x⁵+x+1 → bits (x⁵…x⁰) = 100011. g=x³+x+1 → 1011 (deg 3). Division = repeated "leading-term ÷ leading-term, XOR back". Subtract = XOR.

Round 1 — kill x⁵:

Leading: x⁵÷x³ = x² (first quotient term).
Multiply back: x²·g = x⁵+x³+x² → 6-bit 101100 (x⁵=100000, x³=001000, x²=000100; XOR = 101100).
XOR into dividend: 100011 ⊕ 101100 = 001111 = x³+x²+x+1. (Bit by bit: 1⊕1=0, 0⊕0=0, 0⊕1=1, 0⊕1=1, 1⊕0=1, 1⊕0=1.)

Round 2 — kill x³:

Leading of 001111 is x³: x³÷x³ = 1 (second quotient term).
Multiply back: 1·g = x³+x+1 → padded 001011.
XOR: 001111 ⊕ 001011 = 000100 = x². (Columns: x³ 1⊕1=0, x² 1⊕0=1, x¹ 1⊕1=0, x⁰ 1⊕1=0.)
deg(x²)=2 < deg(g)=3 ⇒ stop. Quotient q = x²+1 (101), remainder r = x² (100).

Verify f = q·g + r (watch cancellations): (x²+1)(x³+x+1) = x⁵+x³+x²+x³+x+1. The two x³ terms: 1⊕1=0 cancel ⇒ x⁵+x²+x+1. Add r=x²: x²⊕x²=0 cancel ⇒ x⁵+x+1 = f ✓.

Irreducible check on g (deg 3): any factorisation needs a linear (deg-1) factor, i.e. a root. g(0) = 0+0+1 = 1 ≠ 0. g(1) = 1+1+1 = 3 mod 2 = 1 ≠ 0 (1⊕1=0, 0⊕1=1). No roots ⇒ no linear factor ⇒ g irreducible (for deg ≤3 this test is sufficient).

Q13 — Cyclic non-systematic (7,4) m=1101

Setup: n=7,k=4 ⇒ n−k=3. g=x³+x+1 (1011), deg g=3 ✓, g | X⁷+1. Message m=1101 (MSB=x³): m(x)=x³+x²+0·x+1 = x³+x²+1.

Non-systematic rule: c(x) = m(x)·g(x) — multiply, message hidden.

Row A: x³·g = x³·(x³+x+1) = x⁶+x⁴+x³. (Each term of g shifted by 3.)
Row B: x²·g = x⁵+x³+x². (Shifted by 2.)
Row C: 1·g = x³+x+1. (Unshifted.)

XOR all rows — count 1s per degree (even count cancels, odd survives):

  x⁶ x⁵ x⁴ x³ x² x¹ x⁰
   1  0  1  1  0  0  0   (Row A)
⊕  0  1  0  1  1  0  0   (Row B)
⊕  0  0  0  1  0  1  1   (Row C)
  -------------------
   1  1  1  1  1  1  1

x⁶:1→keep. x⁵:1→keep. x⁴:1→keep. x³: three 1s, 1⊕1⊕1=1→keep. x²:1→keep. x¹:1→keep. x⁰:1→keep.
c(x) = x⁶+x⁵+x⁴+x³+x²+x+1 → codeword 1111111. Message 1101 NOT readable.
Check: c = m·g by construction ⇒ c mod g = 0 ✓ (valid codeword).

Q14 — Cyclic systematic (7,4) m=1101 ★ most likely

Setup: same m=1101 → m(x)=x³+x²+1, same g=x³+x+1 (1011). Systematic rule: shift message by n−k=3, divide by g, append remainder as parity: c(x)=x³m(x)+r(x).

Step 1 — Shift (multiply by x³ = append three zeros): x³·m(x) = x³(x³+x²+1) = x⁶+x⁵+x³ → bits (x⁶…x⁰) = 1101000. (Message 1101 now in the top 4 positions.)

Step 2 — Long divide 1101000 ÷ 1011, four rounds (7-bit alignment, XOR = subtract):

R1: leading x⁶÷x³=x³. x³·g=x⁶+x⁴+x³=1011000. 1101000⊕1011000=0110000 (=x⁵+x⁴).
R2: leading x⁵÷x³=x². x²·g=x⁵+x³+x²=0101100. 0110000⊕0101100=0011100 (=x⁴+x³+x²).
R3: leading x⁴÷x³=x. x·g=x⁴+x²+x=0010110. 0011100⊕0010110=0001010 (=x³+x).
R4: leading x³÷x³=1. 1·g=0001011. 0001010⊕0001011=0000001 = 1.
deg(1)=0 < 3 ⇒ stop. Quotient = x³+x²+x+1 (1111), remainder r = 1 → 3-bit parity 001 (pad to n−k=3 bits).

Step 3 — Append parity (add r): c(x) = x⁶+x⁵+x³ + 1 = x⁶+x⁵+x³+1.

Step 4 — Codeword: positions x⁶…x⁰: 1 1 0 1 0 0 1 = 1101001. First 4 bits = 1101 = message ✓ visible, last 3 = 001 = parity.

Verify: c mod g = (x³m mod g) ⊕ r = 1 ⊕ 1 = 0 ✓.

Same m=1101	Non-sys (Q13)	Systematic (Q14)
Formula	c=m·g	c=x³m+rem(x³m/g)
Codeword	1111111	1101001
Visible?	No	Yes: 1101+001
Check	c mod g=0 ✓	c mod g=0 ✓

Q15 — Convolutional rate + Viterbi step

Setup (Slide-8 code): feedforward, non-systematic, k=1 input bit per step, n=2 output bits per step.

Rate: R = k/n = 1/2 = ½. Each input bit produces 2 coded bits.
Memory/states: memory m=3 delay elements ⇒ states = 2ᵐ = 2² = 4 (00,01,10,11).
Generators: g0=1011 means taps [1,0,1,1] ⇒ poly g0(D)=1+D²+D³. g1=1111 ⇒ g1(D)=1+D+D²+D³. Output pair = (out0,out1) per input.
Branch metric (Hamming distance): for received pair y=(y0,y1) and branch label l=(l0,l1): metric = (y0≠l0)+(y1≠l1), i.e. count differing bits: 00 vs 00→0, vs 01→1, vs 11→2.
Worked ACS example from state 00, received y=00:
- Input-0 branch label 00: compare 0-vs-0 same, 0-vs-0 same ⇒ metric 0 (0+0).
- Input-1 branch label 11: 0-vs-1 differ, 0-vs-1 differ ⇒ metric 2 (1+1).
- Add to path metrics, compare 0 vs 2, select smaller ⇒ keep input-0 path, discard input-1 at this state.
ACS general rule (per state, per step): two predecessor paths merge → add branch metric to each path metric → compare the two sums → keep (survivor) the smaller, discard the larger. Tie ⇒ either (arbitrary).
Traceback: after last bit, pick best final-state path metric, follow survivor pointers backwards to read off input sequence u. Example: u=10111 → v=(11,01,00,01,01,01,00,11) (given label sequence).