fix: retain fresh screen frames and gate sharing startup
Drain ImageReader continuously while retaining only one latest image, release replaced buffers, snapshot acquisition time before encoding, and keep Share disabled until service binding finishes. Pin hosted acceptance to the verified additive server1.2 source1041693. Pass137 JVM tests, lint/all package builds, fresh APK inspection and TC22 ordinary-UID forged-input rejection. Independent follow-up review found no concrete issue. Paired server consumer/provenance and all15 browser cases pass. Actual OS consent/device capture remains unverified. Co-Authored-By: OpenAI GPT-6.1 (Codex) <noreply@openai.com> Signed-off-by: James Maes <james@kof22.com>
This commit is contained in:
1 parent
fbf9a30a51
commit
e04fdb2d21
8 files changed
+121
-14
No files matched your search
@@ -78,7 +78,7 @@ jobs:
|
||||
timeout-minutes: 40
|
||||
env:
|
||||
QQQ_CORE_SHA: ecf0f550a0357cec6387c2b54efca128a405a629
|
||||
SCANNER_SERVER_SHA: b14d925b741e815d1cfa4f62301f19207f25b03f
|
||||
SCANNER_SERVER_SHA: 10416938724c9663d66f7848ff2b1a63fe635794
|
||||
MAVEN_OPTS: -XX:ActiveProcessorCount=2 -Xmx768m
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
|
||||
|
||||
@@ -4,13 +4,17 @@ The optional `DEVICE_SCREEN` mode requires the additive mobile-scanners1.2 contr
|
||||
|
||||
The initial source checkpoint passes135 JVM tests with no failures/errors/skips, scanner debug/release lint, and debug/unsigned-release/E2E/QA plus native-test package builds. Its paired server passes39 tests and six real-HTTP demo tests using qualified core `ecf0f550`, with no artifact installation and unchanged historical1.0/1.1 API baselines. The frozen client diff received parent and fresh independent static review: all nine inventoried changed files inspected, no reportable findings. The sealed security scan is `ea8a20a8-8a16-48d8-9c58-a9376387fb60`; its tool reports4,884,517 aggregate tokens across two threads. This is a changed-file static review, not a new full-repository audit or device acceptance.
|
||||
|
||||
The initial candidate hashes are:
|
||||
The refined candidate passes137 JVM tests, debug/release lint and all package builds. A separate independent review covered its four-file image-buffer/binding follow-up, with no concrete issue found; the original sealed scan remains unchanged and describes only its original bytes. The TC22 ordinary-UID/distinct-signer regression passes on the refined QA APK: eight forged broadcasts add zero mutations while one manual positive control saves. Physical trigger and projection consent remain unverified.
|
||||
|
||||
All four refined APKs pass fresh SDK DEX/manifest/resource inspection:131 notice entries/nine full license texts, unchanged reviewed native binaries, zero bundled EMDK implementation classes, test-only ingress excluded from production variants, backup exclusions intact, and unsigned release cleartext/debugging disabled. The paired published server `10416938724c9663d66f7848ff2b1a63fe635794` also passes a fresh isolated consumer qualification:195 hashed runtime JARs,2,441 unique QQQ classes, zero duplicates,39 server/six demo tests and all15 record/bulk/admin browser tests, with no shared Maven replacement. Hosted client CI is pinned to that exact source.
|
||||
|
||||
The refined candidate hashes are:
|
||||
|
||||
| APK | SHA-256 |
|
||||
| --- | --- |
|
||||
| Debug | `27A7B6D27A9B2439FE67BABB8950FA9E0EA13099D94383DBE340FB838BBA6DD2` |
|
||||
| Unsigned release | `C80D2012096621D3654242EAB5B4119BF6DC9AF107EEA9FABF52505B5C957A70` |
|
||||
| QA | `2822E45CA9FE39F47B7B620E630342C68E03E8753827CE04BBDD814370003C42` |
|
||||
| E2E | `F6422F428511C7F1F6C8670D90CFEF20F0476696C21B8BD19948C762386F970B` |
|
||||
| Debug | `F3AF638E34AA0B6FB44D789362BED2838202EDAF681DF775F2A5FAD3AB3E9FAA` |
|
||||
| Unsigned release | `69FF9A2F1D7A0A766682994E393D1DE9E3B10CA28D1ECD7239D43E7FB8C57B8C` |
|
||||
| QA | `8F44EFB9A025F0E9457057E25844852BC83B0823B87B5D830AB93BD7962B80EB` |
|
||||
| E2E | `CCB447F8F4121E0FBCF6B8DD5A82784DACCBAF2AF839A10127B6E4CE517CDFBF` |
|
||||
|
||||
Actual OS consent, notification visibility, fresh pixels on a static screen, other-app/background capture, lock, rotation/recreation, administrator disconnect and forced logout remain required device acceptance. A projection may exist locally for up to30seconds awaiting a viewer; uploads require a current matching-mode lease. Binding UI and frame-queue freshness refinements are separate follow-up work; later bytes require their own verification. These APKs are not a production release and have not replaced the operator's normal application.
|
||||
Actual OS consent, notification visibility, fresh pixels on a static screen, other-app/background capture, lock, rotation/recreation, administrator disconnect and forced logout remain required device acceptance. A projection may exist locally for up to30seconds awaiting a viewer; uploads require a current matching-mode lease. Binding stays disabled through service startup; continuous draining retains only one latest image, released on replacement/resize/stop. These APKs are not a production release and have not replaced the operator's normal application.
|
||||
@@ -1,5 +1,9 @@
|
||||
# Session state
|
||||
|
||||
## November 1 — refined device-screen client and published paired server
|
||||
|
||||
The refined client passes137 JVM tests, all package/lint gates, fresh four-APK inspection and the TC22 ordinary-UID scan-ingress regression. Continuous image draining retains one latest buffer; repeated Share is blocked through service binding. Fresh independent follow-up review found no concrete issue. The source-pinned1.2 server `1041693` is published and passes39 server/six demo/all15 browser tests with strict195-JAR/2,441-class provenance in a cloned private Maven cache. Hosted client CI now targets that exact source. No actual projection consent/physical trigger is proved; the normal APK remains unchanged and its running demo remains qualified1.1. See [candidate scope and hashes](DEVICE-SCREEN-ACCEPTANCE.md).
|
||||
|
||||
## November 1 — initial consent-based device-screen checkpoint
|
||||
|
||||
The optional device-screen client and paired1.2 server pass initial compilation and automated gates:135 Android JVM tests, lint/all package builds,39 server and six real-HTTP demo tests. Parent and fresh independent static review found no reportable client issue across the nine changed-file inventory; actual consent and projection behavior remain unverified. See [scope, hashes and remaining acceptance](DEVICE-SCREEN-ACCEPTANCE.md). The normal operator APK and qualified1.1 demo remain separate. Frame freshness/binding refinements and final native acceptance are still required.
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# TODO
|
||||
|
||||
- [x] Compile/review additive consent-based device-screen client and paired1.2 server:137 JVM, package/lint/APK inspection, ordinary-UID TC22 scan-input rejection,39 server/six demo/all15 browser cases and strict provenance pass. Actual projection acceptance remains separate.
|
||||
- [ ] Execute opt-in actual Android consent, notification, Home/background pixels, Stop, lease expiry, lock/rotation and forced-session-end acceptance on the refined device-screen candidate.
|
||||
- [ ] Resolve CircleCI job 140's three unfocused-Activity Back failures using the new bounded window-owner diagnostics; retain real Back and the five-second guard. CodeQL on `6845abc` passed, but native CI is not fully green.
|
||||
- [ ] Repeat physical EMDK acceptance after the unsuccessful first-stage QA attempt preserved in `C:/Users/james/.cache/qqq-tc22-physical-attempt-20261101-0210`; no physical trigger proof is established. Preserve the 120-second operator stage bound and explicit timeout reporting.
|
||||
- [x] Resolve normal backend ownership and perform the reviewed API 1.9 upgrade with fresh backup, original-column/data checks, recovery logins and live account validation; preserve the operator APK. Private verification confirms two devices and three scans retained.
|
||||
|
||||
@@ -19,6 +19,7 @@ import android.graphics.PixelFormat
|
||||
import android.hardware.display.DisplayManager
|
||||
import android.hardware.display.VirtualDisplay
|
||||
import android.media.ImageReader
|
||||
import android.media.Image
|
||||
import android.media.projection.MediaProjection
|
||||
import android.media.projection.MediaProjectionManager
|
||||
import android.os.Binder
|
||||
@@ -43,6 +44,8 @@ class DeviceScreenCaptureService : Service()
|
||||
private var projection: MediaProjection? = null
|
||||
private var display: VirtualDisplay? = null
|
||||
private var reader: ImageReader? = null
|
||||
private val latestImage = LatestScreenImage<Image>()
|
||||
private var latestCapturedAt = ""
|
||||
private var stopped: (() -> Unit)? = null
|
||||
private var foregroundReady = false
|
||||
private var screenReceiverRegistered = false
|
||||
@@ -104,7 +107,7 @@ class DeviceScreenCaptureService : Service()
|
||||
granted.registerCallback(callback, handler)
|
||||
val (width, height) = displaySize()
|
||||
val (scaledWidth, scaledHeight) = scaledSize(width, height)
|
||||
reader = ImageReader.newInstance(scaledWidth, scaledHeight, PixelFormat.RGBA_8888, 2)
|
||||
reader = newReader(scaledWidth, scaledHeight)
|
||||
display = requireNotNull(granted.createVirtualDisplay("QQQ consented device screen", scaledWidth, scaledHeight,
|
||||
resources.configuration.densityDpi, DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR, reader!!.surface, null, handler))
|
||||
ContextCompat.registerReceiver(this, stopReceiver, IntentFilter(Intent.ACTION_SCREEN_OFF), ContextCompat.RECEIVER_NOT_EXPORTED)
|
||||
@@ -129,8 +132,10 @@ class DeviceScreenCaptureService : Service()
|
||||
if(owner != grantOwner || projection == null || lease.captureMode != CaptureMode.DEVICE_SCREEN) return@withContext null
|
||||
val capturedProjection = projection
|
||||
if(getSystemService(KeyguardManager::class.java).isDeviceLocked) { finishProjection(); return@withContext null }
|
||||
val image = reader?.acquireLatestImage() ?: return@withContext null
|
||||
val bitmap = try {
|
||||
val image = latestImage.image ?: return@withContext null
|
||||
val capturedAt = latestCapturedAt
|
||||
// The main-thread listener cannot replace this image while its pixels are copied.
|
||||
val bitmap = run {
|
||||
val plane = image.planes.single()
|
||||
if(plane.pixelStride != 4 || plane.rowStride < image.width * 4 || plane.rowStride.toLong() * image.height > 8388608L)
|
||||
return@withContext null
|
||||
@@ -151,7 +156,7 @@ class DeviceScreenCaptureService : Service()
|
||||
copied.copyPixelsFromBuffer(packed)
|
||||
copied
|
||||
} catch(exception: Exception) { copied.recycle(); throw exception }
|
||||
} finally { image.close() }
|
||||
}
|
||||
try {
|
||||
if(bitmap.width > lease.maxWidth || bitmap.height > lease.maxHeight) return@withContext null
|
||||
withContext(Dispatchers.Default) {
|
||||
@@ -160,7 +165,7 @@ class DeviceScreenCaptureService : Service()
|
||||
val bytes = ByteArrayOutputStream()
|
||||
bitmap.compress(Bitmap.CompressFormat.JPEG, quality, bytes)
|
||||
if(bytes.size() <= lease.maxBytes) {
|
||||
frame = ScreenFrame(bytes.toByteArray(), bitmap.width, bitmap.height, Instant.now().toString(), CaptureMode.DEVICE_SCREEN)
|
||||
frame = ScreenFrame(bytes.toByteArray(), bitmap.width, bitmap.height, capturedAt, CaptureMode.DEVICE_SCREEN)
|
||||
break
|
||||
}
|
||||
}
|
||||
@@ -196,7 +201,8 @@ class DeviceScreenCaptureService : Service()
|
||||
if(reader?.width == targetWidth && reader?.height == targetHeight) return
|
||||
val previous = reader
|
||||
try {
|
||||
val replacement = ImageReader.newInstance(targetWidth, targetHeight, PixelFormat.RGBA_8888, 2)
|
||||
latestImage.close()
|
||||
val replacement = newReader(targetWidth, targetHeight)
|
||||
reader = replacement
|
||||
current.surface = replacement.surface
|
||||
current.resize(targetWidth, targetHeight, resources.configuration.densityDpi)
|
||||
@@ -204,6 +210,24 @@ class DeviceScreenCaptureService : Service()
|
||||
finally { runCatching { previous?.close() } }
|
||||
}
|
||||
|
||||
/** Drain continuously so first demand gets current pixels; encode only on leased polls. */
|
||||
private fun newReader(width: Int, height: Int): ImageReader
|
||||
{
|
||||
// One retained image plus two free slots lets acquireLatestImage discard older queued frames.
|
||||
return ImageReader.newInstance(width, height, PixelFormat.RGBA_8888, 3).also { created ->
|
||||
created.setOnImageAvailableListener({ source ->
|
||||
if(source === reader && projection != null) {
|
||||
try {
|
||||
source.acquireLatestImage()?.let { image ->
|
||||
latestImage.replace(image)
|
||||
latestCapturedAt = Instant.now().toString()
|
||||
}
|
||||
} catch(exception: Exception) { finishProjection() }
|
||||
}
|
||||
}, handler)
|
||||
}
|
||||
}
|
||||
|
||||
private fun finishProjection()
|
||||
{
|
||||
val callback = stopped
|
||||
@@ -216,6 +240,8 @@ class DeviceScreenCaptureService : Service()
|
||||
foregroundReady = false
|
||||
if(screenReceiverRegistered) { runCatching { unregisterReceiver(stopReceiver) }; screenReceiverRegistered = false }
|
||||
runCatching { display?.release() }; display = null
|
||||
runCatching { latestImage.close() }
|
||||
latestCapturedAt = ""
|
||||
runCatching { reader?.close() }; reader = null
|
||||
projectionCallback?.let { callback -> runCatching { prior?.unregisterCallback(callback) } }
|
||||
projectionCallback = null
|
||||
|
||||
@@ -35,6 +35,8 @@ internal class DeviceScreenSharing(private val context: Context)
|
||||
private set
|
||||
var pending by mutableStateOf(false)
|
||||
private set
|
||||
var connecting by mutableStateOf(false)
|
||||
private set
|
||||
var message by mutableStateOf("")
|
||||
private set
|
||||
private val consentAttempt = ScreenConsentAttempt()
|
||||
@@ -61,6 +63,7 @@ internal class DeviceScreenSharing(private val context: Context)
|
||||
return
|
||||
}
|
||||
requireNotNull(data)
|
||||
connecting = true
|
||||
val serviceIntent = Intent(context, DeviceScreenCaptureService::class.java)
|
||||
val binding = object : ServiceConnection {
|
||||
override fun onServiceConnected(name: ComponentName, binder: IBinder)
|
||||
@@ -73,6 +76,7 @@ internal class DeviceScreenSharing(private val context: Context)
|
||||
if(consentAttempt.current(ticket)) { stop(); onSystemStop() }
|
||||
}
|
||||
if(active) onStarted()
|
||||
connecting = false
|
||||
if(!active) { stop(); message = "Device screen sharing could not start. Try again from the app." }
|
||||
}
|
||||
private fun disconnected() { if(connection === this) { stop(); onSystemStop() } }
|
||||
@@ -96,6 +100,7 @@ internal class DeviceScreenSharing(private val context: Context)
|
||||
consentAttempt.invalidate()
|
||||
active = false
|
||||
pending = false
|
||||
connecting = false
|
||||
val previous = service
|
||||
service = null
|
||||
val binding = connection
|
||||
@@ -133,7 +138,8 @@ internal fun DeviceScreenSharingControl(sharing: DeviceScreenSharing, controller
|
||||
ContextCompat.checkSelfPermission(context, Manifest.permission.POST_NOTIFICATIONS) == PackageManager.PERMISSION_GRANTED }
|
||||
if(!canNotify) TextButton(onClick = { if(Build.VERSION.SDK_INT >= 33) notificationPermission.launch(Manifest.permission.POST_NOTIFICATIONS) }) {
|
||||
Text("Enable sharing notification")
|
||||
} else TextButton(enabled = !sharing.pending && allowed(), onClick = { disclosure = true }) { Text("Share device screen…") }
|
||||
} else TextButton(enabled = !sharing.pending && !sharing.connecting && allowed(), onClick = { disclosure = true }) { Text("Share device screen…") }
|
||||
if(sharing.connecting) Text("Starting device screen sharing…")
|
||||
if(sharing.message.isNotBlank()) Text(sharing.message)
|
||||
}
|
||||
if(disclosure) AlertDialog(onDismissRequest = { disclosure = false }, title = { Text("Share this device screen?") },
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
/* Copyright (c) 2026 QRunIO. SPDX-License-Identifier: Apache-2.0 */
|
||||
package io.qrun.mobile.scanner
|
||||
|
||||
/** Main-thread owner of one captured image. Replacing or clearing releases the old buffer. */
|
||||
internal class LatestScreenImage<T : AutoCloseable> : AutoCloseable
|
||||
{
|
||||
var image: T? = null
|
||||
private set
|
||||
|
||||
fun replace(next: T?)
|
||||
{
|
||||
if(image === next) return
|
||||
val previous = image
|
||||
image = next
|
||||
previous?.close()
|
||||
}
|
||||
|
||||
override fun close() { replace(null) }
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
/* Copyright (c) 2026 QRunIO. SPDX-License-Identifier: Apache-2.0 */
|
||||
package io.qrun.mobile.scanner
|
||||
|
||||
import org.junit.Assert.*
|
||||
import org.junit.Test
|
||||
|
||||
class LatestScreenImageTest
|
||||
{
|
||||
private class Image : AutoCloseable {
|
||||
var closes = 0
|
||||
override fun close() { closes++ }
|
||||
}
|
||||
|
||||
@Test fun replacingFramesRetainsOnlyTheNewestAndReleasesPreviousBuffers()
|
||||
{
|
||||
val latest = LatestScreenImage<Image>()
|
||||
val first = Image()
|
||||
val second = Image()
|
||||
latest.replace(first)
|
||||
latest.replace(second)
|
||||
assertSame(second, latest.image)
|
||||
assertEquals(1, first.closes)
|
||||
assertEquals(0, second.closes)
|
||||
latest.close()
|
||||
assertNull(latest.image)
|
||||
assertEquals(1, second.closes)
|
||||
}
|
||||
|
||||
@Test fun repeatedStopAndDuplicateNotificationDoNotCloseAnOwnedBufferTwice()
|
||||
{
|
||||
val latest = LatestScreenImage<Image>()
|
||||
val image = Image()
|
||||
latest.replace(image)
|
||||
latest.replace(image)
|
||||
assertEquals(0, image.closes)
|
||||
latest.close()
|
||||
latest.close()
|
||||
assertEquals(1, image.closes)
|
||||
assertNull(latest.image)
|
||||
val nextSession = Image()
|
||||
latest.replace(nextSession)
|
||||
assertSame(nextSession, latest.image)
|
||||
latest.close()
|
||||
assertEquals(1, nextSession.closes)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user