ink/gestlings

gestlings

dz / ink / gestlings

Summary

Sketches and brainstorms related to the Gestlings project

Node Tree

Nodes

top
content top-level node
children ink/thoughts_2023/gestlings

mouth_rigging
content Mouth Rigging Animations
children 2023_apr05_next_vocal_rigging_milestone (initial planning for mouth rigging milestone), 2023_apr05_vocal_rigging, 2023_aug02_p1_mouth_animations, 2023_aug02_p2_10_point_mouth, 2023_may01_p3_rectangular_deformation, 2023_sep29_p1_mouth_patterns
parents pipeline

planning
content Planning and Scoping
children 2023_apr23_p1_gestku_pain_points, 2023_sep12_p2_whats_left, 2023_sep24_whats_left
parents concepts

2023_apr05_next_vocal_rigging_milestone
content Initial planning for the "mouth rigging" milestone. The goal here was to get gestures produced in Sound to interpolate between to visual mouth states. This would eventually become the moving mouth mechanics seen in "Welcome To Gestleton".
children 2023_apr05_vocal_rigging (planning)
parents mouth_rigging
remarks gfuuqeqed
tags planning, vocal, rigging
image

2023_apr05_vocal_rigging
content Vocal Rigging: a flow chart outlining roughly how vocal rigging could work.
parents mouth_rigging, 2023_apr05_next_vocal_rigging_milestone
remarks giwufqhrf
image

concepts
content Concepts and story development
children 2023_apr10_gestlings_sounds_with_faces (working out "sounds with faces"), 2023_apr20_p2_protogestling_vocab, 2023_apr25_p1_morpheme_notation, 2023_jul03_protogestling, 2023_jul23_p1_gestling_map, 2023_jul23_p2_cauldronia_symbol, 2023_jul24_p1_gestleton, 2023_jul24_p2_the_rings, 2023_jul25_p1_lower_rings, 2023_jul25_p2_shores, 2023_jul26_p1_depths, 2023_jul26_p2_hall_of_was, 2023_jul26_p3_linear_story, 2023_jun05_what_now, 2023_jun13_what_now, 2023_jun22_gestling_biology, 2023_jun22_gestlings_time_and_space, 2023_jun23_finish_gestlings, 2023_jun28_cauldron_to_springs, 2023_jun28_great_springs, 2023_may01_p1_sound_physiology, 2023_may21_gestlings_what_now, 2023_nov03_gestlings_narrative, 2023_nov05_gestleton, 2023_nov06_gestling_story, 2023_sep11_what_does_junior_say, 2023_sep12_p1_junior_outline, planning

2023_apr10_gestlings_sounds_with_faces
content "Gestlings Are Sounds With Faces That Talk About Themselves" I think this was first time I put to paper what I wanted the Gestlings to be about. All the way up to this point, what I wanted the Gestlings to be was muddy. Initially, it was going to be a set of etude studies in Gesture Synthesis, similar to what I did with Sporthlings. Sometime in December 2022 or so I started messing with the idea that these could be etudes with the ability to introduce themselves... characters. However, this page really scoped out the project boundaries for me.
parents concepts
remarks guauuskud
tags planning, sounds, faces, sporthlings
image

pipeline
content The Gestling Pipeline
children speech, 2023_apr11_gestling_components_mechanics, 2023_apr17_gestlings_scrolling_text, 2023_apr27_p3_notation_parsing_pipeline, 2023_aug02_p3_avatar_components, 2023_jul13_p1_shapemorf, 2023_jul13_p2_generating_shapes_shapemorf, 2023_jul15_p2_shapemorf_node, 2023_jul16_p1_tubeshape, 2023_jul19_tubesculpt_shapemorf, 2023_jul28_p1_gestling_pipeline, 2023_jul28_p2_gestling_architecture, 2023_jul28_p3_time_left, 2023_jun27_tubular, 2023_may01_p2_asset_driven_design, 2023_oct07_the_second_gestling, 2023_sep26_inspire, 2023_sep29_p4_avatar_animations, messagebox, mouth_rigging

2023_apr11_gestling_components_mechanics
content Gestling Components and Mechanics: At this point, it was important to scope out this project. What was it going to be? What was I going to build? What kind of time do I have? This flow chart attempts to map out the infrastructure of Gestlings.
parents pipeline
remarks gsfidhkjj
tags components, planning
image

2023_apr17_gestlings_scrolling_text
content Scrolling text logic. I was working out some pseucode for how a message box could work. I'm sure this was a good starting point, but the scrolling text I ended up making was far more sophisticated because I wanted more kinematic text (slow down, speedup, pause, bigger text, backspace, etc).
parents pipeline
remarks gqqiljhie
tags scrolling, text, messagebox
image

2023_apr18_conductor_pause
content Conductor Pause Mechanic. An issue I had to solve was how to synchronize message box scrolling text with the corresponding performed Gesture. One idea I had was to be able to pause the Conductor signal and have it wait for the text to complete. Of course, this mechanic would only work if the message box was longer than the phrase. I did not end up using this idea.
parents gesture_mechanics
remarks gddoosipq
tags conductor, pause, gesture
image

2023_apr20_p2_protogestling_vocab
content Protogestling Vocab: here I am thinking about the first Gestling and planning how to efficient build it out. The protogestling was the test proof-of-concept Gestling to come before Junior was made.
children 2023_apr20_p3_protogestling_vocab_tasks (action plan protogestling)
parents concepts
remarks gquroaqii
tags protogestling, planning
image

2023_apr20_p3_protogestling_vocab_tasks
content Planning out actionable tasks for building what would become the Protogestling. Some parts I was very anxious about because they seemed so large ("vocab building: why does this feel daunting?")
parents 2023_apr20_p2_protogestling_vocab
remarks gopaphrdf
image

2023_apr23_p1_gestku_pain_points
content Gestkus were initially a plan for me to build daily gesture etudes. Small bits of code that I could make quickly and explore ideas. This never really took off the way I liked it. Gestkus were tedious to build, required lots of boilerplate and adhoc code. Here I do a postmortem and reflect on the situation.
parents planning
remarks gidqrdfrl
tags gestku, postmortem
image

2023_apr23_p2_morpheme
content Morpheme: Data format, DSL, notation. It seems like at this point the Morpheme was mostly a thing? I had grand ideas of building compact an expressive structures to notate and convey Morphemes. It turns out, I was thinking too small. Morphemes used to create gestling vocabularies have dozens of parameters to consider, and you really need another abstraction to manage it all. At some point, I would think about partial morphemes and morpheme operations, which would be a step in the right direction. Wish this was easier to do.
parents gesture_mechanics
remarks gsrafrffj
image

2023_apr25_p1_morpheme_notation
content Concept sketches for morpheme notation building. Early on I knew I wanted the Gestlings to have notation, but I believe composers need a score. The problem I wanted to solve here was how to build a good notation system around define morpheme structures and morpheme paths. Here I am thinking about the lower-level details, which is why there are number systems here.
children 2023_apr26_gesture_path_notation (thinking about individual paths in a morpheme)
parents concepts, gesture_mechanics
remarks gfdpllril
tags notation, concept, morpheme
image

2023_apr26_gesture_path_notation
content Concept sketches for gesture path notation. Thinking mostly about numbers here. In this iteration, I'm trying to simplify the glyphs to simple patterns that could fit in compact spaces. I don't think the writing on the right means anything. I think I was just testing the glyphs out. Mainly I wanted to be able to have a handwritten version as well as a blocky digitized version.
parents symbols, 2023_apr25_p1_morpheme_notation, gesture_mechanics
remarks geupofief
tags notation, concept, numbers, gesture
image

2023_apr26_morpheme_operations
content It quickly become clear to me that morpheme structures would benefit from some kind of alegebraic operations. A full morpheme can have dozens of paths in it, so it's much more elegant to define a morpheme out of smaller morphemes (called partial morphemes), rather than to explicitly define each path. At the time of writing (decemeber 2023), I only ended up implementing a handful of these operations, and of those, I've only needed to use "overlay", which I now called "merge".
parents gesture_mechanics
remarks ghppfawqk
tags morpheme, operations, algebra
image

symbols
content symbols
children 2023_apr23_p3_symbol_driven_live_coding, 2023_apr24_symbol_driven_design, 2023_apr25_p2_making_symbols, 2023_apr25_p4_symbol_crafting_workflow, 2023_apr26_gesture_path_notation, 2023_apr27_p1_symbol_design, 2023_jul04_bitrune, 2023_jul16_p2_blocky6, 2023_jul23_p2_cauldronia_symbol, 2023_jun06_p1_notate_16, 2023_jun27_bitrune_revisited, 2023_nov09_p1_toni_writing_system

2023_apr27_p1_symbol_design
content Symbol Design: how to construct compelling symbols? Notation and symbols are an important part to the Gestling aesthetic, so in this brainstorm I attempted to think about what goes into making interesting symbols, given the constraints that the symbol would be 16x16 pixels.
parents symbols
remarks gsjqpdias
tags symbols, brainstorm
image

2023_apr25_p4_symbol_crafting_workflow
content At this point, I was working out how I would build an interactive symbol-based editor for notating Gestling phrases and vocabulary. When I wrote this, I think I may not have been thinking about using the monome grid as a display yet. the "3x3 ortho interface" refers to using a 3x3 grid found on a numberpad as a chord/keyshape input system. This workflow would eventually evolve into the tool I built called "bitrune" which I use to write gestling phrases.
parents symbols
remarks giojisaws
tags symbols, workflow, bitrune, planning
image

2023_apr25_p2_making_symbols
content making symbols. Here I'm creating a diagram for all the things I'd need to implement a working symbol system. The symbol format (uxn uf2), lookup tables, documentation, keyshaps for the 3x3 ortholinear key input interface, etc. Most of this got implemented. I had this notion of using grammars in LPEG, but as of writing (Decemeber 2023), I've settled for simpler parsers written in lua, as it ended up being simpler and more practical for my immediate needs (phrase building).
parents symbols
remarks gpukhkdwj
tags symbols
image

2023_apr27_p3_notation_parsing_pipeline
content Notation Parsing Pipeline: this is a high level map outlining how I imagined symbols could be parsed into meaningful data. This is very close to how Bitrune works. The display is actually the grid (framebuffer treats Grid as a very low-res 1-bit display). Grammar generators like Lemon/LPeg never got used (December 2023).
parents pipeline
remarks gurprwlrf
tags symbols, notation, parsing, pipeline, bitrune
image

2023_may01_p1_sound_physiology
content Sound Physiology. This is perhaps one of the earliest written instances where I started using the terminology of "physiology" to describe how a Gestling sound is constructed, rather than something like "synth patch". In this diagram, I'm showing the hierarchy of components that make up an overarching-physiology: mechanisms, morphemes, vocabulary, articulator. The terminology would end up varying a bit by the time I got around to implementing this months later, but it's very similar. Physiology now refers to the underlying patch (I was thinking of it as a mechanism before), mechanisms control the physiology (what I called an articulator here), and morphemes are used to form a working sonic vocabulary that is used to produce gestling phrases.
parents concepts
remarks garkuuhpp
tags physiology, mechanism, articulator, morpheme
image

2023_apr23_p3_symbol_driven_live_coding
content I was/am pretty determined to make Gestlings very "symbol" focused, and I knew I wanted something more specialized than a general text editor to create sequences of symbols. The problem with text editors is that it's very hard to break habits and associations with text. My hope was that symbols would encourage new thinking. I also wanted a new interface that didn't feel like a typical keyboard, because I know that too well. I was still thinking about a traditional display on a buffer. It would take some crazy "what-ifs" and "just-do-it"s to get me thinking about working with the grid.
parents symbols
remarks grkiedrsj
tags bitrune, symbols, brainstorm
image

2023_apr24_symbol_driven_design
content At the time, I was pretty frustrated with all the code I had to put on my screen in order to make even the smallest insequential sounds. Gestkus, intended to be small daily etudes, got weighed down by boilerplate lua code. So, I had this idea of trying to reduce the creative space down to a compact and expressive set of symbols and work with that. This is what Tidal does really well. It scopes the creative domain to patterns rhythms, and instrument selection, and abstracts away everything else. At the time of writing (December 2023), I've made some progress with this creating notation systems for gestling phrases, but it takes a lot of prep work and is by no means a streamlined process.
children 2023_may01_p2_asset_driven_design (similar, also inspired by lua-fatigue)
parents symbols
remarks gsppdooqj
tags brainstorm, symbols, design
image

2023_may01_p2_asset_driven_design
content Still driven crazy by the mess of ad-hoc Lua code I made for myself, I started thinking about the elegance of just saving things to disk as simple data structures. No weird code incantations or configuration, just load the data up, and it works. In other words, work incrementally to build components that make a gestling, and then save those components to disk as data. In this example, I was thinking about all the bits that would go into generating a gesture path, but this key shift of thinking ended up being a big part of how the Gestling Pipeline wors
parents 2023_apr24_symbol_driven_design, pipeline
remarks gpeuupsou
tags pipeline, asset
image

2023_may01_p3_rectangular_deformation
content Rectangular Deformation for Mouths. These were some early concept sketches for creating a simple mouth to animate. As can be seen, I was considering about the possibility of using SDFs to build this, and I did end up building this mouth model using SDFs, with an added ability to morph it into a circle. You can see this mouth in action with things like "Welcome to Gestleton" and "Junior".
parents mouth_rigging
remarks gfeeelhaj
tags concept, mouth, SDF
image

2023_may21_gestlings_what_now
content Gestlings: what now? Trying to concisely define what it is the Gestlings are all about. Transcription: Gestlings are sounds with faces. Gestlings have personality. Gestlings have something to say. Faces have mouths. Mouths produce sound. Gestlings live in Gestleton, on the planet Cauldronia, formerly known as Cavernius. Meta: this project that attempts to bootstrap itself into a world of (with) narrative. Mouths emit language (asemic), sound structures with the veneer of musical speech (speeched music). Speeched music is produced from an articulated mechanism. The articulation system is Gesture. The mechanism is a graph.
parents concepts
remarks gapqiolru
tags brainstorm, narrative, speeched, music, gestleton
image

2023_jun05_what_now
content Here I'm trying to collect and focus my thoughts to make some progress with the Gestlings. A lot of this is very general, but I know at the time I was struggling trying to make the push from "this is a neat technical achievement" to "this is something with an interesting narrative".
parents concepts
remarks glohsaheq
image

gesture_mechanics
content gesture mechanics
children 2023_apr18_conductor_pause, 2023_apr23_p2_morpheme, 2023_apr25_p1_morpheme_notation, 2023_apr26_gesture_path_notation, 2023_apr26_morpheme_operations, 2023_jun10_rescale_path_fit_morphseq (rescale a path to fit a morpheme sequence), 2023_sep07_p1_partial_morpheme_DSL

2023_jun10_rescale_path_fit_morphseq
content Rescale Path to Fit Morpheme Sequence. This is a very jargony phrase. The big-picture problem I was trying to solve here was how to add inflection (prosody curves) to a phrase that a Gestling is saying. A morpheme is mostly equivalent to a "word", and sequence of morphemes makes a phrase. The problem to solve here was how to take a path, a line segment of arbitrary length controlling something like pitch, and stretch or squash it to fit the length of the overal sequence. This logic is how I can do things like make junior sound excited or sad or bored.
parents gesture_mechanics
remarks gpdqddekf
tags morpheme, path
image

2023_jun13_what_now
content What now? Jun 13. Scoping mostly. What to focus on, what not to focus on. I'm thinking about notation, and I'm starting to think more seriously about the so-called "Gestling Pipeline".
parents concepts
remarks gpuujosda
tags pipeline, planning
image

2023_jun22_gestling_biology
content Gestling Biology. Here I was thinking about sound design in terms of biology. I use the term "physiology" here to describe a synth patch, which is a term I still use in the Gestlings.
parents concepts
remarks gkekwejel
tags physiology, biology, brainstorm
image

2023_jun22_gestlings_time_and_space
content Gestlings: time and space. I'm trying to work on narrative here. Sounds are characters that have things to say. So they need a place to say it, and a context. Here I was working on the notion that I the author would work on a program that would eventually bootstrap its way into a world with narrative and lore. Cauldron was where all the core engine code would be made. Springs would be low level assets and things in between code and lore. Then you'd have shores and gestleton where it would be a fully world with narrative. It's a nice idea, but the creation process has ended up being far less elegant than this. So I've ended up scrapping the idea for the most part (Dec 2023). I have no idea what I was getting at with "space" other than physical size I was imagining. Was I thinking about it in the narrative sense? Or scope / amount of work? Nobody knows!
parents concepts
remarks gapdodwde
tags brainstorming
image

2023_jun23_finish_gestlings
content Finish the Gestlings. I really wanted to finish the Gestlings up in 3 months (by October 1st) because I had to focus on other aspects of my life like getting job. This is where I got the idea to limit the Gestlings to 275 hours + 25 hours overtime. I figured that would be about 3 months work. Turns 300 hours takes more than 3 months for me, especially when the hours are logged. It's December 2023, we're at about 220 hours, work is now a trickle, and I'm unsure if I gave myself enough time.
parents concepts
remarks geerwushs
tags planning
image

2023_jun27_bitrune_revisited
content bitrune revisited. "bitrune" is a name I've used for years to describe quite a few things, including some kind of specialized text editor. Here I've decided I wanted it to be a "symbol" editor controllable by knob. I was thinking about symbols in terms of fixed pages called "slabs", which is a neat idea if your display has enough information density. Adapted to the grid, I went with more of a scroll-based approach instead.
parents symbols
remarks gfiipudqd
tags bitrune
image

2023_jun27_tubular
content Planning out a model for tubular. It's a physical vocal tract model based off of my other models, except without the nasal component. I also wanted a way to make the tract size a parameter in centimeters rather than samples, and to have tract shapes use a distinct region model (DRM) so that I could re-use shapes with different sizes. This resizability was what makes "Junior" sound so cute and small!
parents pipeline
remarks gjslfweep
tags tubular, dsp
image

2023_jun28_cauldron_to_springs
content From Cauldron To Springs. Once again attempting to flesh out the idea of bootstrapping into worldbuilding by building an engine (Cauldron), an intermediary zone (springs), and then a world. Hasn't worked out (yet? Dec 2023), but a cool idea.
parents concepts
remarks glqprrris
tags brainstorm, meta, cauldron, springs
image

2023_jun28_great_springs
content Concept art for a "Springs" landing page. Each bubble was to be a clickable portal to another page.
parents concepts
remarks girohkafj
tags concept, springs
image

2023_jul03_protogestling
content Protogestling scoping. Sketching out what the gestling format would look like, how it would sound, etc. It's interesting that I'm calling this a "protogestling" because I think I already made something called a protogestling by this name already. Note the breakdown of components at the bottom. A lot of Gestling planning has been trying to effectively break things into components.
parents concepts
remarks gloshoiwh
tags planning, protogestling
image

2023_jul04_bitrune
content More scoping for the Bitrune symbol editor
parents symbols
remarks ghopjokrr
tags bitrune, planning
image

2023_jul13_p1_shapemorf
content Initial outlining for "Shapemorf", the tool used in Gestlings for smoothly connecting vowel sounds together. Shapemorf is built on top of Tubular, and uses Gesture synthesis via GestVM to interpolate between shapes
parents pipeline
remarks gwworuiol
tags shapemorf, planning, gestvm, gesture, tubular
image

2023_jul13_p2_generating_shapes_shapemorf
content Generating shapes with Shapemorf. There needed to be a way to interactively "sculpt" vocal tract shapes that could be saved and used for later. I wanted to use the Grid and my knob controller (KN01, previously Arc), which is what I had done for past projects. This line of thinking would eventually lead to a tool I built called Tubesculpt.
parents pipeline
remarks guiaehhhp
tags brainstorm, shapemorf, tubesculpt
image

2023_jul15_p2_shapemorf_node
content Outlining the actual components of the Shapemorf Node. Coordinating the gesture synthesizer (GestVM) with the vocal tract model (tubular) involved a lot of moving parts, so it was helpful to draw a visualization of how it all conected together before implementing any code.
parents pipeline
remarks gqphosjda
tags shapemorf, gestvm, tubular, planning
image

2023_jul16_p1_tubeshape
content More detailed outlining for a vocal tract shaping interface using the Grid, numpad, the KN01 knob. I was calling it "tubeshape" when I started doodling, and by the end I was thinking about "tubesculpt" as name. You'll note the "tubesculpt?" as an afterthought that I wrote in the end, which is what I finally ended up calling it.
parents pipeline
remarks gdfrefhlf
tags tubeshape, tubesculpt, planning
image

2023_jul16_p2_blocky6
content Block6 was a really niche symbol set I can up with while building tubesculpt. I needed something that could display on a monome grid, and was really thin so I could put many symbols on it. The idea here was to use these symbols to construct larger symbols. I believe a version of this was used in tubesculpt for an autonaming feature. When you save a shape, there's no interface to actually enter in a name, and there's no room to display the name either. So a name gets auto-generated and represented as a symbol. I added some extra logic to constrain the choices of combinations so it was more aesthetically pleasing, and called that system "Klover". (Update: Klover doesn't use blocky6 rows, it's only 4 bits, but was derived from blocky6).
children 2023_jul16_p3_4bit_SFM_kufic
parents symbols
remarks gafjlruhh
tags blocky6, klover, tubesculpt
image

2023_jul16_p3_4bit_SFM_kufic
content 4bit SFM (sic) using kufic rules. I was working on a notation system called blocky6 where each character was a horiztonal strip of 6 bits that could be stacked together to make a larger character. I wanted to restrict the choices so that the resulting glyph conformed to basic kufic rules. I realized these choices could be structured as a Finite State Machine (FSM). I started with 4 bits FSMs, which I then realized could be broken down in 2-bit FSMs. This system would go on to be called Klover. Instead of 6 bits, Klover would be only 4 bits using this FSM system.
parents 2023_jul16_p2_blocky6
remarks gwedilsir
tags klover, blocky6, kufic, kuf, fsm
image

2023_jul19_tubesculpt_shapemorf
content More interface planning for tubesculpt (formerly known as shapemorf). I've added more detail in about interface functionality here.
parents pipeline
remarks gslkkahqu
tags planning, tubesculpt
image

2023_jul23_p1_gestling_map
content Gestling Map: a map of the planet Cauldronia, the homeworld of the Gestlings.
parents concepts
remarks gjlwjjfqi
tags maps, concept
image

2023_jul23_p2_cauldronia_symbol
content After I drew the map of Cauldronia with the giant Gestletree sticking out of it, I wanted to make an abstract symbol of it that could work as a tiled mosaic. This is the symbol seen at the end of "Welcome to Gestleton"
parents concepts, symbols
remarks gdfjflfju
tags cauldronia, symbols
image

2023_jul24_p1_gestleton
content Concept sketch of the Gestleton map.
parents concepts
remarks gwuadkojs
tags maps, concept
image

2023_jul24_p2_the_rings
content Concept sketch and notes for the Rings District map. I think I realized halfway through I wanted to break the Rings District up into the "Upper Rings" and "Lower Rings"
parents concepts
remarks gllhuidda
tags concept, maps
image

2023_jul25_p1_lower_rings
content Concept sketch and notes for the Lower Rings District.
parents concepts
remarks gorhsdlfe
tags concept, brainstorming, maps
image

2023_jul25_p2_shores
content Concept sketch and notes for the Shores.
parents concepts
remarks gqdorsewp
tags maps, concept
image

2023_jul26_p1_depths
content Concept sketch and notes for the Depths.
parents concepts
remarks gsraphpkf
tags maps, concept
image

2023_jul26_p2_hall_of_was
content Concept sketch and notes for the Hall of Was.
parents concepts
remarks gisdoheru
tags maps, concept
image

2023_jul26_p3_linear_story
content My attempt at summarizing the Gestlings as a linear story. Non-linear adventures are more fun, but linear stories are faster to make, and time has been a factor while making the Gestlings.
parents concepts
remarks gkelqhufw
tags planning, story
image

2023_jul28_p1_gestling_pipeline
content One thing I started thinking about more and more was this notion of a "Gestling Pipeline", or the process I'd use to manufacture Gestlings. This was my attempt to try and refine/focus this idea.
parents pipeline
remarks gdijrdeps
tags planning
image

2023_jul28_p2_gestling_architecture
content I was starting to think about building the first Gestling at this point, and so I had to start working out what all the actual components were going to be. This was helpful to see. There are many moving parts to a Gestling.
children 2023_jul28_p3_time_left (Figured out component list. how much time to build them?)
parents pipeline
remarks gjoaeaqfd
tags brainstorm, architecture
image

2023_jul28_p3_time_left
content Initially, I was planning to be done with the Gestlings by Novemeber 2023. This was my attempt at trying to map out the timeline and assess how much time I had left. This was coordinated with the gestling architecture planning I did on this same day.
parents 2023_jul28_p2_gestling_architecture, pipeline
remarks gdoilaueo
tags planning, timeline
image

2023_aug02_p1_mouth_animations
content Building on previous ideas related to animation, this model aimed to build a mouth that was smooth and around. I looked at a few cartoon drawing books for reference, as well as a few cartoon characters from Adventure Time (Lumpy Space Princess in particular). I was all-in on SDFs at this point because of how easy it was to construct smooth organic shapes using them.
parents mouth_rigging
remarks gkdefwksr
tags mouth, animation, brainstorm
image

2023_aug02_p2_10_point_mouth
content A concept I had for a 10-point mouth design. As of December 2023, this has yet to be built.
parents mouth_rigging
remarks gphlrquai
image

2023_aug02_p3_avatar_components
content At this point, I was getting ready to actually implement the "avatar" component of the first gestling (Junior), so I needed to think about what I actually needed to build.
parents pipeline
remarks gqkswsdaa
tags avatar, planning
image

messagebox
content Message Box
children 2023_aug30_p1_message_box_component, 2023_aug30_p3_message_mechanics, 2023_aug31_static_queue
parents pipeline

2023_aug30_p1_message_box_component
content Outlinging what the message box was going to look like before implmenting it.
parents messagebox
remarks gowllqfqu
tags messagebox, planning
image

speech
content Speech
children 2023_aug30_p2_speech_component, 2023_oct13_tony_vocab_taxonomy, 2023_sep05_p1_speech_timing, 2023_sep05_p2_phrase_structure, 2023_sep07_p2_junior_physiology
parents pipeline

2023_aug30_p2_speech_component
content Development of the First Gestling was broken up into components, and at this point I was starting to think about the actual sound components. It was helpful to start thinking of the sound compositions I was making as "Speech", and this started bringing me full circle to about 2 years prior when I began thinking about these concepts of Gesture. There is a video of Sir Ian McKellen workshopping the opening lines of the Merchant of Venice, and he's being directed by John Barton to say them in different ways. I thought "what if music could be like this". You'll see the opening lines written in the top-righthand corner "in sooth I know not why I am so sad".
parents speech
remarks giwkwjdkj
tags speech, planning
image

2023_aug30_p3_message_mechanics
content Message Box Mechanics: I was trying to think about how this message box was going to do, and how I was going to do it to implement it.
children 2023_aug31_static_queue
parents messagebox
remarks gpawrkeoa
tags brainstorm, planning, messagebox
image

2023_aug31_static_queue
content While planning the message box, I had this notion of generating a static event queue from a parsed script which would serve as the representation of the message box animation. This is the data structure used in the Inspire utility.
parents messagebox, 2023_aug30_p3_message_mechanics
remarks gkoasosqr
image

2023_sep05_p1_speech_timing
content Big problem to solve: how to coordinate speech sound with message boxes? This was the initial outline for the proposed solution. The core idea here is to take sound phrases, bind one or more of them to a messagebox of text, and then use some Gesture magic to stretch the phrases to fit the duration of the messagebox. I had a few notions for how I was going to solve this problem, and this ended up being the winning solution.
parents speech
remarks grqkjadoa
tags brainstorm, speech, timing
image

2023_sep05_p2_phrase_structure
content Phrase Structure: I was trying work out the concept of "phrases" as they'd appear when synced with the message box. There were still many things to work out.
parents speech
remarks gkjquwrio
tags brainstorm, speech, phrase
image

2023_sep07_p1_partial_morpheme_DSL
content An idea to create a notation language specifically for creating partial morphemes and operations on partial morphemes to construct vocabularies. As of December 2023, no work has been done on this. I ended up just going with the path of least resistance and using lua tables. Practical, though at times clunky and inelegant.
parents gesture_mechanics
remarks gslqooqur
tags morpheme, DSL, notation
image

2023_sep07_p2_junior_physiology
content Junior Physiology: At this point, I was trying to flesh out what the actual underlying physiology for "Junior" was going to look like. What was the signal graph going to look like? What were the points of articulation? etc. etc.
parents speech
remarks gwejijpse
tags planning, physiology, junior
image

2023_sep11_what_does_junior_say
content What does Junior say? This some brainstorming done for the second trailer video. In contrast with the "Welcome to Gestleton Video", this one was going to focus on the textbox and the phrasing system for speech. I wanted the dialogue to reference as much of the Gestling world as possible, and I also wanted to showcase the various features of the Junior physiology. What emerged (Junior writing a letter home from camp) was a result of this process.
children 2023_sep12_p1_junior_outline
parents concepts
remarks gpaiqfwof
tags brainstorm, junior
image

2023_sep12_p1_junior_outline
content More concrete outlining for the script for Junior Writes Home.
parents concepts, 2023_sep11_what_does_junior_say
remarks giirfhjud
tags planning, outlining, junior
image

2023_sep12_p2_whats_left
content 2023_sep12_p2_whats_left
parents planning
remarks gpsfrfowi
image

2023_sep24_whats_left
content More planning, thinking about what's left with the Gestlings (still a lot to do)
parents planning
remarks gwhedpwej
tags planning
image

2023_sep26_inspire
content Inspire: this is the name of the utility in charge of generating videos from a script. It looks like I was just trying to fit it all together here.
parents pipeline
remarks gowjdihou
tags planning, inspire
image

2023_sep29_p1_mouth_patterns
content mouth patterns: more detailed thinking about coordinating mouth movements with gesture.
parents mouth_rigging
remarks gulwqaroq
tags brainstorm, mouth
image

2023_sep29_p4_avatar_animations
content Avatar Animations. Here I am trying to break down what kind of facial animations I wanted. As of December 2023, I haven't implemented any of this yet. But I really want it.
parents pipeline
remarks glwlaweqo
tags animation, brainstorm, avatar
image

2023_oct07_the_second_gestling
content With Junior (the first gestling) out of the way, I needed to start thinking about what I was going to do for the second gestling. I articulate my thoughts here.
parents pipeline
remarks gfewffhqp
tags brainstorm
image

2023_oct13_tony_vocab_taxonomy
content Toni Vocab Taxonomy. I wanted to make sure that the points of articulation I'd make were perceptually signficiant. So, instead of jumping in and working out a signal flow graph out of unit generators, I thought about the kinds of sounds I wanted the structure to make and organized it as a tree-like structure.
parents speech, gestlings/toni_vocab_taxonomy/top
remarks gakwahofk
tags taxonomy, toni, brainstorm
image

2023_nov03_gestlings_narrative
content Gestlings Narrative: at this point, I needed to start thinking practically. What is the underlying narrative of the Gestlings, and how many Gestlings am I going to need to make to convey that narrative? A mixture of brainstorm and planning because the number of gestlings gives a coarse approximation of how much is left to be done. As of December 2023, I don't have a solid number for how many gestlings I want to make.
parents concepts
remarks gojeerusq
tags brainstorm, planning
image

2023_nov05_gestleton
content This is an attempt at outlining the narrative of the gestlings in terms of level design where each level asks a different layer of the story. Level 1, gestleton, asks the "what" questions (what is gestleton? What are gestlings? what is Inspiration?, etc). Level 2 asks "Who is Cap?", where we learn who Cap is, and that they are missing. Level 3 aims to figure out why Cap is missing. Etc, etc.
parents concepts
remarks glohrwqfk
tags brainstorm, story
image

2023_nov06_gestling_story
content The Gestling Story: I just wanted to see if I could sumarize the current version of story in as little words as possible, with the idea being that eventually these would be disperssed to Gestling characters somehow as plot devices.
parents concepts
remarks gwjfjhshi
tags brainstorm, story
image

2023_nov09_p1_toni_writing_system
content Concept sketches for the writing system used by Toni.
children 2023_nov09_p2_toni_glyphs_conversion_to_6height (convert handwritten symbols to blocky pixel art)
parents symbols
remarks glwruriww
tags concept, toni, symbols
image

2023_nov09_p2_toni_glyphs_conversion_to_6height
content Converting sketched out symbols for toni and attempting to convert them to a pixel font 6 pixels high so it fits on Bitrune.
parents 2023_nov09_p1_toni_writing_system
remarks ghiuajjfo
tags symbols, bitrune, toni, concept
image

2023_jun06_p2_notate_16
content More work on a grid-friendly human readable symbolset for base16
parents 2023_jun06_p1_notate_16
remarks gdldiihea
tags symbols, base16, grid
image

2023_jun06_p1_notate_16
content How to Notate to 16? When I was thinking more seriously about notation in Gestlings, I felt it was important to come up with a counting system that could be human readable but also small enough to display on a monome grid. Here, I try to think about a base-16 counting system notation.
children 2023_jun06_p2_notate_16
parents symbols
remarks gwoshwjir
tags symbols, base16, grid
image