fix: retain fresh screen frames and gate sharing startup

Drain ImageReader continuously while retaining only one latest image,
release replaced buffers, snapshot acquisition time before encoding,
and keep Share disabled until service binding finishes. Pin hosted
acceptance to the verified additive server1.2 source1041693.

Pass137 JVM tests, lint/all package builds, fresh APK inspection and
TC22 ordinary-UID forged-input rejection. Independent follow-up review
found no concrete issue. Paired server consumer/provenance and all15
browser cases pass. Actual OS consent/device capture remains unverified.

Co-Authored-By: OpenAI GPT-6.1 (Codex) <noreply@openai.com>
Signed-off-by: James Maes <james@kof22.com>
This commit is contained in:
james.maesandOpenAI GPT-6.1 committed 2026-10-31 21:51:12 -05:00
1 parent fbf9a30a51
commit e04fdb2d21
8 files changed
+121 -14

No files matched your search

+1 -1
View File
@@ -78,7 +78,7 @@ jobs:
timeout-minutes: 40
env:
QQQ_CORE_SHA: ecf0f550a0357cec6387c2b54efca128a405a629
SCANNER_SERVER_SHA: b14d925b741e815d1cfa4f62301f19207f25b03f
SCANNER_SERVER_SHA: 10416938724c9663d66f7848ff2b1a63fe635794
MAVEN_OPTS: -XX:ActiveProcessorCount=2 -Xmx768m
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
+10 -6
View File
@@ -4,13 +4,17 @@ The optional `DEVICE_SCREEN` mode requires the additive mobile-scanners1.2 contr
The initial source checkpoint passes135 JVM tests with no failures/errors/skips, scanner debug/release lint, and debug/unsigned-release/E2E/QA plus native-test package builds. Its paired server passes39 tests and six real-HTTP demo tests using qualified core `ecf0f550`, with no artifact installation and unchanged historical1.0/1.1 API baselines. The frozen client diff received parent and fresh independent static review: all nine inventoried changed files inspected, no reportable findings. The sealed security scan is `ea8a20a8-8a16-48d8-9c58-a9376387fb60`; its tool reports4,884,517 aggregate tokens across two threads. This is a changed-file static review, not a new full-repository audit or device acceptance.
The initial candidate hashes are:
The refined candidate passes137 JVM tests, debug/release lint and all package builds. A separate independent review covered its four-file image-buffer/binding follow-up, with no concrete issue found; the original sealed scan remains unchanged and describes only its original bytes. The TC22 ordinary-UID/distinct-signer regression passes on the refined QA APK: eight forged broadcasts add zero mutations while one manual positive control saves. Physical trigger and projection consent remain unverified.
All four refined APKs pass fresh SDK DEX/manifest/resource inspection:131 notice entries/nine full license texts, unchanged reviewed native binaries, zero bundled EMDK implementation classes, test-only ingress excluded from production variants, backup exclusions intact, and unsigned release cleartext/debugging disabled. The paired published server `10416938724c9663d66f7848ff2b1a63fe635794` also passes a fresh isolated consumer qualification:195 hashed runtime JARs,2,441 unique QQQ classes, zero duplicates,39 server/six demo tests and all15 record/bulk/admin browser tests, with no shared Maven replacement. Hosted client CI is pinned to that exact source.
The refined candidate hashes are:
| APK | SHA-256 |
| --- | --- |
| Debug | `27A7B6D27A9B2439FE67BABB8950FA9E0EA13099D94383DBE340FB838BBA6DD2` |
| Unsigned release | `C80D2012096621D3654242EAB5B4119BF6DC9AF107EEA9FABF52505B5C957A70` |
| QA | `2822E45CA9FE39F47B7B620E630342C68E03E8753827CE04BBDD814370003C42` |
| E2E | `F6422F428511C7F1F6C8670D90CFEF20F0476696C21B8BD19948C762386F970B` |
| Debug | `F3AF638E34AA0B6FB44D789362BED2838202EDAF681DF775F2A5FAD3AB3E9FAA` |
| Unsigned release | `69FF9A2F1D7A0A766682994E393D1DE9E3B10CA28D1ECD7239D43E7FB8C57B8C` |
| QA | `8F44EFB9A025F0E9457057E25844852BC83B0823B87B5D830AB93BD7962B80EB` |
| E2E | `CCB447F8F4121E0FBCF6B8DD5A82784DACCBAF2AF839A10127B6E4CE517CDFBF` |
Actual OS consent, notification visibility, fresh pixels on a static screen, other-app/background capture, lock, rotation/recreation, administrator disconnect and forced logout remain required device acceptance. A projection may exist locally for up to30seconds awaiting a viewer; uploads require a current matching-mode lease. Binding UI and frame-queue freshness refinements are separate follow-up work; later bytes require their own verification. These APKs are not a production release and have not replaced the operator's normal application.
Actual OS consent, notification visibility, fresh pixels on a static screen, other-app/background capture, lock, rotation/recreation, administrator disconnect and forced logout remain required device acceptance. A projection may exist locally for up to30seconds awaiting a viewer; uploads require a current matching-mode lease. Binding stays disabled through service startup; continuous draining retains only one latest image, released on replacement/resize/stop. These APKs are not a production release and have not replaced the operator's normal application.
+4
View File
@@ -1,5 +1,9 @@
# Session state
## November 1 — refined device-screen client and published paired server
The refined client passes137 JVM tests, all package/lint gates, fresh four-APK inspection and the TC22 ordinary-UID scan-ingress regression. Continuous image draining retains one latest buffer; repeated Share is blocked through service binding. Fresh independent follow-up review found no concrete issue. The source-pinned1.2 server `1041693` is published and passes39 server/six demo/all15 browser tests with strict195-JAR/2,441-class provenance in a cloned private Maven cache. Hosted client CI now targets that exact source. No actual projection consent/physical trigger is proved; the normal APK remains unchanged and its running demo remains qualified1.1. See [candidate scope and hashes](DEVICE-SCREEN-ACCEPTANCE.md).
## November 1 — initial consent-based device-screen checkpoint
The optional device-screen client and paired1.2 server pass initial compilation and automated gates:135 Android JVM tests, lint/all package builds,39 server and six real-HTTP demo tests. Parent and fresh independent static review found no reportable client issue across the nine changed-file inventory; actual consent and projection behavior remain unverified. See [scope, hashes and remaining acceptance](DEVICE-SCREEN-ACCEPTANCE.md). The normal operator APK and qualified1.1 demo remain separate. Frame freshness/binding refinements and final native acceptance are still required.
+2
View File
@@ -1,5 +1,7 @@
# TODO
- [x] Compile/review additive consent-based device-screen client and paired1.2 server:137 JVM, package/lint/APK inspection, ordinary-UID TC22 scan-input rejection,39 server/six demo/all15 browser cases and strict provenance pass. Actual projection acceptance remains separate.
- [ ] Execute opt-in actual Android consent, notification, Home/background pixels, Stop, lease expiry, lock/rotation and forced-session-end acceptance on the refined device-screen candidate.
- [ ] Resolve CircleCI job 140's three unfocused-Activity Back failures using the new bounded window-owner diagnostics; retain real Back and the five-second guard. CodeQL on `6845abc` passed, but native CI is not fully green.
- [ ] Repeat physical EMDK acceptance after the unsuccessful first-stage QA attempt preserved in `C:/Users/james/.cache/qqq-tc22-physical-attempt-20261101-0210`; no physical trigger proof is established. Preserve the 120-second operator stage bound and explicit timeout reporting.
- [x] Resolve normal backend ownership and perform the reviewed API 1.9 upgrade with fresh backup, original-column/data checks, recovery logins and live account validation; preserve the operator APK. Private verification confirms two devices and three scans retained.
@@ -19,6 +19,7 @@ import android.graphics.PixelFormat
import android.hardware.display.DisplayManager
import android.hardware.display.VirtualDisplay
import android.media.ImageReader
import android.media.Image
import android.media.projection.MediaProjection
import android.media.projection.MediaProjectionManager
import android.os.Binder
@@ -43,6 +44,8 @@ class DeviceScreenCaptureService : Service()
private var projection: MediaProjection? = null
private var display: VirtualDisplay? = null
private var reader: ImageReader? = null
private val latestImage = LatestScreenImage<Image>()
private var latestCapturedAt = ""
private var stopped: (() -> Unit)? = null
private var foregroundReady = false
private var screenReceiverRegistered = false
@@ -104,7 +107,7 @@ class DeviceScreenCaptureService : Service()
granted.registerCallback(callback, handler)
val (width, height) = displaySize()
val (scaledWidth, scaledHeight) = scaledSize(width, height)
reader = ImageReader.newInstance(scaledWidth, scaledHeight, PixelFormat.RGBA_8888, 2)
reader = newReader(scaledWidth, scaledHeight)
display = requireNotNull(granted.createVirtualDisplay("QQQ consented device screen", scaledWidth, scaledHeight,
resources.configuration.densityDpi, DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR, reader!!.surface, null, handler))
ContextCompat.registerReceiver(this, stopReceiver, IntentFilter(Intent.ACTION_SCREEN_OFF), ContextCompat.RECEIVER_NOT_EXPORTED)
@@ -129,8 +132,10 @@ class DeviceScreenCaptureService : Service()
if(owner != grantOwner || projection == null || lease.captureMode != CaptureMode.DEVICE_SCREEN) return@withContext null
val capturedProjection = projection
if(getSystemService(KeyguardManager::class.java).isDeviceLocked) { finishProjection(); return@withContext null }
val image = reader?.acquireLatestImage() ?: return@withContext null
val bitmap = try {
val image = latestImage.image ?: return@withContext null
val capturedAt = latestCapturedAt
// The main-thread listener cannot replace this image while its pixels are copied.
val bitmap = run {
val plane = image.planes.single()
if(plane.pixelStride != 4 || plane.rowStride < image.width * 4 || plane.rowStride.toLong() * image.height > 8388608L)
return@withContext null
@@ -151,7 +156,7 @@ class DeviceScreenCaptureService : Service()
copied.copyPixelsFromBuffer(packed)
copied
} catch(exception: Exception) { copied.recycle(); throw exception }
} finally { image.close() }
}
try {
if(bitmap.width > lease.maxWidth || bitmap.height > lease.maxHeight) return@withContext null
withContext(Dispatchers.Default) {
@@ -160,7 +165,7 @@ class DeviceScreenCaptureService : Service()
val bytes = ByteArrayOutputStream()
bitmap.compress(Bitmap.CompressFormat.JPEG, quality, bytes)
if(bytes.size() <= lease.maxBytes) {
frame = ScreenFrame(bytes.toByteArray(), bitmap.width, bitmap.height, Instant.now().toString(), CaptureMode.DEVICE_SCREEN)
frame = ScreenFrame(bytes.toByteArray(), bitmap.width, bitmap.height, capturedAt, CaptureMode.DEVICE_SCREEN)
break
}
}
@@ -196,7 +201,8 @@ class DeviceScreenCaptureService : Service()
if(reader?.width == targetWidth && reader?.height == targetHeight) return
val previous = reader
try {
val replacement = ImageReader.newInstance(targetWidth, targetHeight, PixelFormat.RGBA_8888, 2)
latestImage.close()
val replacement = newReader(targetWidth, targetHeight)
reader = replacement
current.surface = replacement.surface
current.resize(targetWidth, targetHeight, resources.configuration.densityDpi)
@@ -204,6 +210,24 @@ class DeviceScreenCaptureService : Service()
finally { runCatching { previous?.close() } }
}
/** Drain continuously so first demand gets current pixels; encode only on leased polls. */
private fun newReader(width: Int, height: Int): ImageReader
{
// One retained image plus two free slots lets acquireLatestImage discard older queued frames.
return ImageReader.newInstance(width, height, PixelFormat.RGBA_8888, 3).also { created ->
created.setOnImageAvailableListener({ source ->
if(source === reader && projection != null) {
try {
source.acquireLatestImage()?.let { image ->
latestImage.replace(image)
latestCapturedAt = Instant.now().toString()
}
} catch(exception: Exception) { finishProjection() }
}
}, handler)
}
}
private fun finishProjection()
{
val callback = stopped
@@ -216,6 +240,8 @@ class DeviceScreenCaptureService : Service()
foregroundReady = false
if(screenReceiverRegistered) { runCatching { unregisterReceiver(stopReceiver) }; screenReceiverRegistered = false }
runCatching { display?.release() }; display = null
runCatching { latestImage.close() }
latestCapturedAt = ""
runCatching { reader?.close() }; reader = null
projectionCallback?.let { callback -> runCatching { prior?.unregisterCallback(callback) } }
projectionCallback = null
@@ -35,6 +35,8 @@ internal class DeviceScreenSharing(private val context: Context)
private set
var pending by mutableStateOf(false)
private set
var connecting by mutableStateOf(false)
private set
var message by mutableStateOf("")
private set
private val consentAttempt = ScreenConsentAttempt()
@@ -61,6 +63,7 @@ internal class DeviceScreenSharing(private val context: Context)
return
}
requireNotNull(data)
connecting = true
val serviceIntent = Intent(context, DeviceScreenCaptureService::class.java)
val binding = object : ServiceConnection {
override fun onServiceConnected(name: ComponentName, binder: IBinder)
@@ -73,6 +76,7 @@ internal class DeviceScreenSharing(private val context: Context)
if(consentAttempt.current(ticket)) { stop(); onSystemStop() }
}
if(active) onStarted()
connecting = false
if(!active) { stop(); message = "Device screen sharing could not start. Try again from the app." }
}
private fun disconnected() { if(connection === this) { stop(); onSystemStop() } }
@@ -96,6 +100,7 @@ internal class DeviceScreenSharing(private val context: Context)
consentAttempt.invalidate()
active = false
pending = false
connecting = false
val previous = service
service = null
val binding = connection
@@ -133,7 +138,8 @@ internal fun DeviceScreenSharingControl(sharing: DeviceScreenSharing, controller
ContextCompat.checkSelfPermission(context, Manifest.permission.POST_NOTIFICATIONS) == PackageManager.PERMISSION_GRANTED }
if(!canNotify) TextButton(onClick = { if(Build.VERSION.SDK_INT >= 33) notificationPermission.launch(Manifest.permission.POST_NOTIFICATIONS) }) {
Text("Enable sharing notification")
} else TextButton(enabled = !sharing.pending && allowed(), onClick = { disclosure = true }) { Text("Share device screen…") }
} else TextButton(enabled = !sharing.pending && !sharing.connecting && allowed(), onClick = { disclosure = true }) { Text("Share device screen…") }
if(sharing.connecting) Text("Starting device screen sharing…")
if(sharing.message.isNotBlank()) Text(sharing.message)
}
if(disclosure) AlertDialog(onDismissRequest = { disclosure = false }, title = { Text("Share this device screen?") },
@@ -0,0 +1,19 @@
/* Copyright (c) 2026 QRunIO. SPDX-License-Identifier: Apache-2.0 */
package io.qrun.mobile.scanner
/** Main-thread owner of one captured image. Replacing or clearing releases the old buffer. */
internal class LatestScreenImage<T : AutoCloseable> : AutoCloseable
{
var image: T? = null
private set
fun replace(next: T?)
{
if(image === next) return
val previous = image
image = next
previous?.close()
}
override fun close() { replace(null) }
}
@@ -0,0 +1,46 @@
/* Copyright (c) 2026 QRunIO. SPDX-License-Identifier: Apache-2.0 */
package io.qrun.mobile.scanner
import org.junit.Assert.*
import org.junit.Test
class LatestScreenImageTest
{
private class Image : AutoCloseable {
var closes = 0
override fun close() { closes++ }
}
@Test fun replacingFramesRetainsOnlyTheNewestAndReleasesPreviousBuffers()
{
val latest = LatestScreenImage<Image>()
val first = Image()
val second = Image()
latest.replace(first)
latest.replace(second)
assertSame(second, latest.image)
assertEquals(1, first.closes)
assertEquals(0, second.closes)
latest.close()
assertNull(latest.image)
assertEquals(1, second.closes)
}
@Test fun repeatedStopAndDuplicateNotificationDoNotCloseAnOwnedBufferTwice()
{
val latest = LatestScreenImage<Image>()
val image = Image()
latest.replace(image)
latest.replace(image)
assertEquals(0, image.closes)
latest.close()
latest.close()
assertEquals(1, image.closes)
assertNull(latest.image)
val nextSession = Image()
latest.replace(nextSession)
assertSame(nextSession, latest.image)
latest.close()
assertEquals(1, nextSession.closes)
}
}