📡 23ECE467 · Quiz 2 Masterguide

Given

• X ∈ {0, 1}, P_X = [½, ½] (uniform)
• Channel matrix M = [[¾, ¼], [¼, ¾]] — each row sums to 1

Steps

1. P_Y = P_X · M = [½·¾ + ½·¼,  ½·¼ + ½·¾] = [½, ½]
2. Joint = scale row i of M by P(x_i): row 0 → ½·[¾, ¼] = [⅜, ⅛]; row 1 → ½·[¼, ¾] = [⅛, ⅜]

Joint P(X, Y)

H — entropies

• H(X) = 1 bit (uniform), H(Y) = 1 bit (uniform)
• Each row of M is Bernoulli(¼): H(row) = H_b(¼) ≈ 0.811
• H(Y|X) = ½·H_b(¼) + ½·H_b(¼) ≈ 0.811 bits
• H(X, Y) = H(X) + H(Y|X) ≈ 1 + 0.811 = 1.811 bits

I — mutual information

Given

• X ∈ {0, 1}, P_X = [¼, ¾] (not uniform)
• Channel matrix M = [[1, 0], [½, ½]] — row 0 is noiseless, row 1 is fair coin

Steps

1. Joint row 0 = ¼ · [1, 0] = [¼, 0]
2. Joint row 1 = ¾ · [½, ½] = [⅜, ⅜]
3. P_Y = column sums of the joint = [¼ + ⅜, 0 + ⅜] = [⅝, ⅜]

Joint P(X, Y)

H — entropies

• H(X) = −¼·log₂¼ − ¾·log₂¾ ≈ 0.811 bits
• H(Y) = −⅝·log₂⅝ − ⅜·log₂⅜ ≈ 0.954 bits
• H(Y|X) = ¼·H([1, 0]) + ¾·H([½, ½]) = ¼·0 + ¾·1 = 0.75 bits
• H(X, Y) = H(X) + H(Y|X) ≈ 0.811 + 0.75 = 1.561 bits

I — mutual information

Given

• Binary Symmetric Channel with crossover probability ε = ¼
• Capacity is the max of I(X; Y) over P(X) — achieved at P(0) = P(1) = ½

Steps

1. H_b(¼) ≈ 0.811 bits
2. C = 1 − H_b(ε) = 1 − H_b(¼) = 1 − 0.811

Answer

Given

• Binary Symmetric Channel with crossover probability ε = ⅛
• Capacity is the max of I(X; Y) over P(X) — achieved at P(0) = P(1) = ½

Steps

1. H_b(⅛) ≈ 0.544 bits
2. C = 1 − H_b(ε) = 1 − H_b(⅛) = 1 − 0.544

Answer

Given

• Binary Erasure Channel with erasure probability γ (α) = ¼
• BEC never flips a bit — it only erases it; capacity is max at P(0) = P(1) = ½

Steps

1. C = 1 − γ = 1 − ¼ = ¾ bit/use
2. For n = 1000 uses: max info bits = n · C = 1000 × ¾ = 750

Answer

Step 1 — convert dB to a linear power ratio (the step that usually goes wrong):

Step 2 — Shannon–Hartley capacity:

Step 3 — evaluate: log₂4 = 2, so C = 4000 × 2 = 8000 bps.

Step 1 — convert dB to a linear power ratio:

Step 2 — Shannon–Hartley capacity:

Step 3 — approximate: 1024 = 2¹⁰, so log₂(1024) = 10 and 1001 is just under it:

Step 1 — test the additive structure ({0,1}, +₂) as a group:

1. Closure: 0+0 = 0, 0+1 = 1, 1+1 = 0 — every sum stays in {0,1}. ✓
2. Identity: 0, since x + 0 = x for both elements. ✓
3. Inverse: each element is its own inverse (0+0 = 0, 1+1 = 0). ✓
4. Associative + commutative: +₂ is XOR, so both hold. ✓

⇒ ({0,1}, +₂) is an Abelian group.

Step 2 — test the multiplicative structure ({0,1}, ×) as a group:

1. Closure: products stay in {0,1}. ✓
2. Identity: 1, since x · 1 = x. ✓
3. Inverse: 0 has none — 0 · x = 0 ≠ 1 for every x. ✗

⇒ ({0,1}, ×) is NOT a group.

Step 3 — field verdict: drop 0; the non-zero set {1} is a group under ×, and × distributes over +₂ on all of {0,1}:

Step 1 — addition table (mod 3):

 + | 0  1  2
---+---------
 0 | 0  1  2
 1 | 1  2  0
 2 | 2  0  1

Step 2 — multiplication table (mod 3):

 × | 0  1  2
---+---------
 0 | 0  0  0
 1 | 0  1  2
 2 | 0  2  1

Step 3 — inverses: 1⁻¹ = 1 (since 1·1 = 1); 2⁻¹ = 2 (since 2·2 = 4 ≡ 1 mod 3).

Step 4 — solve 2x + 1 = 0 (mod 3):

Check: 2(1) + 1 = 3 ≡ 0 (mod 3). ✓

Step 1 — write both polynomials as GF(2) coefficient columns (x³ x² x¹ x⁰):

   x³ x² x¹ x⁰
   -----------
   1  0  1  1    (x³+x+1)
⊕  0  1  1  1    (x²+x+1)
   -----------
   1  1  0  0    (x³+x²)

Step 2 — XOR each column: 1⊕0 = 1, 0⊕1 = 1, 1⊕1 = 0, 1⊕1 = 0.

GF(2) addition is XOR — no carries, coefficients mod 2.

Steps (all arithmetic over GF(2): coefficients mod 2, add = XOR):

1. Message / polys: A(x)=x²+1 → 101 · B(x)=x+1 → 11
2. Distribute: x²·(x+1) = x³+x² and 1·(x+1) = x+1
3. Add (XOR, no carries): (x³+x²) ⊕ (x+1) — no coefficient cancels, all four terms survive
4. Result: x³+x²+x+1 → codeword 1111

Steps (polynomial long division over GF(2), subtract = XOR):

1. Divide this: f(x)=x⁵+x+1 → 100011  ÷  g(x)=x³+x+1 → 1011
2. First term: x⁵÷x³ = x² · x²·g = x⁵+x³+x² · f ⊕ x²g = x³+x²+x+1
3. Second term: x³÷x³ = 1 · 1·g = x³+x+1 · (x³+x²+x+1) ⊕ g = x²
4. Quotient: q(x)=x²+1 → 101  ·  Remainder: r = x² → 100

Irreducible check on g: g(0)=1, g(1)=1+1+1=1 (mod 2) → no linear factor. deg g = 3, so any non-trivial factor would be linear → g is irreducible.

Steps 1–4 — non-systematic encoding (c = m·g, message hidden in the product):

1. Message: m = 1101 → m(x) = x³+x²+1
2. Generator: g(x) = x³+x+1 → 1011, deg g = n−k = 3, and g | (X⁷+1)
3. Multiply: c(x) = m(x)·g(x) = (x³+x²+1)(x³+x+1)
x³·g = x⁶+x⁴+x³ · x²·g = x⁵+x³+x² · 1·g = x³+x+1
   XOR (x³ appears 3× → survives): c(x) = x⁶+x⁵+x⁴+x³+x²+x+1
4. Codeword: c = 1111111 — the message 1101 is not readable in it

Steps 1–4 — systematic encoding (message visible up front, parity appended):

1. Message, shifted by n−k = 3: m = 1101 → m(x) = x³+x²+1
x³·m(x) = x⁶+x⁵+x³ → 1101000
2. Divide by g: (x⁶+x⁵+x³) ÷ (x³+x+1)
x³·g = x⁶+x⁴+x³ → rem x⁵+x⁴ · x²·g = x⁵+x³+x² → rem x⁴+x³+x² · x·g = x⁴+x²+x → rem x³+x · 1·g = x³+x+1 → rem 1
   Remainder: r = 1 → parity 001
3. Append parity: c(x) = x³m(x) + r(x) = x⁶+x⁵+x³+1
4. Codeword: c = 1101001 — message 1101 readable up front, parity 001 at the tail

Both codewords are valid — different codewords of the same (7,4) code. The systematic form is the examinable one: always work out rem(xⁿ⁻ᵏm / g) by long division, never by spotting.

Steps (Slide-8 code: rate ½, feedforward, non-systematic):

1. Parameters: k=1, n=2, memory m=2 → R = k/n = ½, states = 2ᵐ = 4
2. Generators: g0 = 1011 → g0(D) = 1+D²+D³ · g1 = 1111 → g1(D) = 1+D+D²+D³
3. Branch metric: for received pair y and branch label l, metric = Hamming(y, l)
4. ACS (add–compare–select), per state: add each incoming branch metric to its path metric → two paths merge per state → compare → keep the smaller survivor, discard the other
5. Traceback: after the last bit, start from the best final state and walk back the survivor pointers to read off u

u = 10111 → v = (11,01,00,01,01,01,00,11). If two paths tie, either may be kept (Viterbi arbitrary-tie rule).