tagged: spark scribe

prototype one was feature complete for our base platform to experiment with… except that it slowed to a crawl in it’s first real-world session. prototype two has the engineering work to maintain a snappy drawing experience and UI. this hasn’t made the drawing itself performant, rather it has made the engine responsive regardless of the drawing performance. it keeps the act of drawing syncronous along with the rest of UI, while pushing the updating of the whole canvas in the background.

prototype one looked something like this, the standard fill-in-the-placeholder function to draw into the view –

	override func draw(_ rect: CGRect)

the prototype two equivalent looks like this. it’s… a lot more complicated.

	// Drawing canvas for the live stroke, which changes during stroking given prediction etc.
	private var activeStrokeLayer: CALayer!
	private var drawingLayerDelegate: DrawingLayerDelegate!
	    
	// Drawing canvas for the completed strokes; async, accumulating, and other sophistry
	private var completedStrokesLayer: CALayer!
	private var layerDelegate: LayerDelegate!
	private var completedStrokesQueue: DispatchQueue!
	private var completedStrokesBlocks: [DispatchWorkItem]!
	private var completedStrokesContext: CGContext!
	
	// Draw methods to abstract what’s going on behind the scenes
	func drawCompletedStrokes(onlyAdd: [Stroke]? = nil)
	func drawActiveStroke()
	
	// When receiving a new set of strokes, if any stroke from the old set is missing from the new, we need to completely redraw. Conversely, new strokes can be drawn on top of the existing canvas.
	func diff(_ oldElements: [Stroke], _ newElements: [Stroke]) -> diffResult

oh, and the timestamps on the audio timeline are nicely formatted now.

diary | 12 aug 2019 | tagged: spark scribe · live illustration · code · pervasive media studio

friend camille aubry was having thoughts about how her scribing practice could develop. i got an email that said “digital scribe?”; as she suspected, that’s something i have thoughts about too.

live illustration is something i’ve long played with – e.g. the concept and tooling for kinetxt – and in drawing interactions i have a recent iPad code-base for drawing-and-media. i’d been scheming that something in this vein would be interesting for future engaging-audiences work.

taking camille’s question as a prompt, i started coding. a base prototype was needed. first, to verify the basics: can camille draw with an iPad and its pencil? is the 10.5” size i have too limited? can my drawing code compete with a single marker pen and paper? with that established, we’d then have a platform to iteratively develop and test ideas.

so: spark scribe, an iOS app for illustrators documenting live events.

the core idea i have for the app is to combine the act of drawing with a dictaphone. by timestamping the strokes to the audio recording, the context for what is being drawn can always be returned to. plus, taking the user interface approach of drawing interactions, fluid and artful interactions should be possible.

despite the drawing interactions codebase, this is entirely new.

• timeline implemented with UIScrollView and CATiledLayer (the bespoke approach in drawing interactions was quick to code and effective in use, but this approach gives e.g. the inertial scrolling an iOS user would expect)
• persistence using native Swift serialisation and GRDB (‘because one sometimes enjoys a sharp tool’; Core Data didn’t work out)
• drawing code based on the newer apple example Leveraging Touch Input for Drawing Apps (it’s unoptimised but a better foundation than the prior example used in drawing interactions)

prototype one was used for the pervasive media studio’s lunchtime talk. it soon became clear that the drawing code needed work as the app slowed to a crawl. this can’t have made drawing with it easy for camille, let alone it being a new tool. the drawing example code being ‘unoptimsed’ refers to a few things, but mostly that it redraws every stroke, on every change. this was fine to verify the look, and tested fine in sessions of a few minutes at my desk, where the total number of strokes stays low. but is now proven not fine for the quantities of strokes that start to build up scribing a real talk. now i knew this would have to be addressed at some point, so it’s just brought that engineering task from the near-future to now. plus, thanks to this scribing session, i have real-world data i can load-test the app with.

Features: Prototype One
- Drawing engine
	- A3 sized canvas with pan and zoom
	- Pen tool with undo/redo
- Audio engine
	- Records, plays back talk with timeline
	- On playback, scrolling timeline displays illustration as drawn at that point in time
- Persistence
	- Persists illustration + audio
	- Imports and exports
		- UTI `net.sparklive.scribe.archive`
		- extension `.sparkscribe` 
		- format is a package wrapping
			- `strokes.json`
			- `audio.m4a`

diary | 26 jul 2019 | tagged: spark scribe · live illustration · code · pervasive media studio