I did some tests with layering a set of visuals over a pre-recorded performance from the PreViz final Mock-Up video. I played this video mock up while triggering events for 4 different layered visuals. The screen capture from this performance was than layered with the existing performance footage and associated audio track. While the screen capture is not that great, the concept seems to be working out.
My plan for the final execution will including refining these visuals a bit more before attempting to do this type of performance live (but in a controlled setting). I will project the same type of performance that is pictured here over top of these two performers. This will become an element of the final ProZeuxis demonstration video. The other elements will include footage of me operating the touch surface, as well as the screen capture describing how to create the performance composition using the system.
Ok - there may look like there is mostly data related changes going on here - but it is all for a good cause toward what ProZeuxis always intended to be. That is, a toolbox where the VJ can plug visual nodes together in interesting ways with tools that feed these nodes with input from the performance space (such as audio and motion tracking). Until now - we did not have this "pluggin/linking" capability, to make the visuals performance driven in any capacity.
Guess what? - now we do. Lets just run down the major development milestones achieved this week, and what is and is not planned for the Nov 10 Demo:
Node-To-Node Linking (partially complete) Last round - I implemented the menus and interactions that allow linking between a midi controller and a parameter node. This has been taken a step further, as has always been the intent, to allow a user to link one parameter node to another directly. This interaction is working but there are some important points to note:
- Linking between nodes is only implemented for boolean values at this current time - this was the proof of concept. Number, point. color data still needs to be worked out
- The menu interaction to link one parameter to another is done by dragging an icon from the PZNode menu and dropping it on the icon of a compatible node
- Other ways to create a node-to-node link through the param panel box and the ability to manage these links using the [L] icon menu is still yet to be worked on, and at this point, will not be included for Nov10, but may still find its way into the final.
Metronome Data Node Created - A Data providing node called "MetroGnome" was created for the purpose of providing a Triggering type of boolean data that can be linked to other nodes (such as enable/ disabling a node, triggering to create a fish, etc)
- The MetroGnome comes with 2 independently configurable "Beat Instance" subnodes. Each one can have its own BPM value set by a slider control.
- As a MetroGnome Beat ticks, an output value toggles back and fourth. This is the boolean value that can be plugged into visuals or other data nodes that would accept this type of value.
- One beat's value can even be plugged into the second beat's enable/disable switch for some interesting results. IE: Beat 1 is set at 220 bmp, and Beat 2 is set at 15 bpm. By plugging Beat 2 into the enable switch of beat one - then plugging Beat 1into the "create fish trigger", we quickly pop out a burst of about 10 fish every 4 seconds.
- The creation of this node proved to be good testing and preparation for another type of data providing node such as an audio input.
Sonia Audio For my first AUDIO related node, I utilized the LiveInput capabilities of the library "Sonia" (Thus the node name).
- My first audio input node works a whole lot like the MetroGnome node. It has the same basic structure that includes an enable/disable switch, a slider value, and an output trigger value. Except in Sonia, out slider value controls a Volume Threshold that the live input is listening for rather than setting a bpm.
- By creating a Sonia Audio node and setting a threshold value for a LiveInput instance, a user can then Link the Input Listener trigger to any boolean param node much like we did with the MetroGnome beat.
- As a first test, I picked on the Create Fish Trigger once again. Now a new fish is created by making sound into the microphone. The sensitivity at which this will occur can be controlled by the Volume Threshold slider. This all seemed to work with much success (sorry this success happened a lot later than would have been possible to document with a demo video)
Nov 10 Demo So now the fun part comes in. I beleive I have enough now to put together a more robust type of demo. The only other thing that I may try to add in terms of data interaction before locking things down for Nov 10 will be the ability to pass other values through node linking so I can output the Volume value as a number as well as the Boolean from my new SoniaAudio node.
- Other than this change, I will work on one or two new visuals to add to the set that will best exploit this new capability. I was looking at a Grid type visual that looks like flashing LED's. The Grid could change in composition by being linked into the audio detection node.
- With this new capability and visuals, I will take the system into a longer space (such as a garage) and captured footage of a filmed demostration "in the round" so to speak. I will have a friend asssist me with the filming and executing of this new demo.
Lately, the project development has been done with the prototype table powered Off and this ReacTivision simulator software turned on. Below:
Essentially this has sped the development work up a good deal, but has left the Physical status of my prototype a bit lacking. For the final presentation, I would like to have a much better table in place. Ideally, it would both look good and function well. So this weekend, I set to work on remedying the table situation with the help of my friend. The following photos chronicle the birth of the ProZeuxis Workstation 1.0.1.
1. Began with a basic frame to house the illuminated plexi touch surface (surface not pictured)
2. Improved mounting bracket for softbox/camera to tripod
3. Surface enclosure closes tightly with cabinet hardware
4. Solution for surface front legs
5. Pre Camera mounting hardware
6. General layout coming together
7. Camera mount (screws into tripod mount through a hole in the softbox covering
8. Mount slides into place
9. Working out first round of design bugs
10. Calibrating camera for ReacTivision / ProZeuxis "road test"
11. Good with tests and some alterations - hanging up for a paint job
The new prototype table that has come out of this week's effort should suit me well for the preparing and gathering of system demonstration footage for the final. The previous table would just not have done the work behind this project any justice. It was only a matter of time before these measures were taken before I could pursue serious demonstrations of ProZeuxis in any venue other than my home.
For the midterm - I have created an instructional type video for the ProZeuxis system. The video, unfortunately, rendered at a file size slightly higher than I could post to the web - so it will not be available here until I have time to save it out in parts or at a different compression.
The video introduces the ProZeuxis workspace and steps a user through configuring two visual nodes. Along the way, various concepts that may not be self explanatory are covered in detail. A narrator explains:
- Parameter Grid, Node Configuration Menu, Midi Input - The design of the selected PZ Node menu - Moving, Closing, and Opening a Parameter Box - How to use "Quick Keys" - Linking a parameter to a midi controller - Linking multiple parameters to the same controller - Overwriting a midi controller link
This set of explanations will be expanded upon for the final defence to include interactions that are not yet implemented. This will include:
- The remaining two menu buttons [L]ink [R]andom
Link Button: - The Link Icon button will control whether or not a Midi input link is Enabled, Disabled, or Not Set - This button will be colored Green, Red, or Gray to represent these 3 options - Pressing the button will toggle between Enabled and Disabled
Random Button: - The random button will open a small menu to allow a user to configure a parameter for random input - The menu will include a Min/Max selection area for that type of parameter input - A Slider for controlling the Frequency that a Random selection should be changed, (Once, every 2000 ms, ect) - Also a linkable trigger input that would allow changing the random value to occur from some stimulus by the user or some other node parameter (such as linking it to a key press - or to a audio beat)
Node-To-Node Linking: - Dragging and dropping a connection between two nodes of compatible parameters. - This will allow for interesting interactions within a composition of visual and other tool nodes
Z-Order Menu (possibly) - The Node Configuration menu will eventually double as a Z-Order menu - This menu would allow a user to change the order in which visuals are drawn (A Layer Manager of sorts)
Other Convenience Buttons - While testing the system, many situations come up where it seems like a function or helper button may be needed. - For example, a button that closes all open parameter boxes - A button that will disable all parameter boxes without closing them - allowing you to freely configure a new node on top of this surface without "disturbing" open touch panels. - A pallet for loading saved colors, values, etc (this may be within a node - but some interface of the core system would need to be written to allow this to work)
I apologize for a somewhat limited amount of sync between the two layers of this demo - some parts synched right - others did not, but the general premise is there.
This demonstrates two newer capabilities of the system. One being the management of parameter panels on the user interface. A use can now close a param panel both through touching the Orbital Node menu or on the param panel top left hand icon. Touching and holding the icon also allows them to dynamically link the opening and closing of the parameter to a midi keyboard key. A key can be linked to any number of panels. Unlinking of panels is not currently worked out - does not mean it cannot be done. This is a low priority feature for the deliverables of the final defense.
Next is the capability to layer, enable/disable, and effect two different Visual Plugin Nodes simultaneously. I show a flowing particle wave visual layered with the Phishstyxx visual. One of the next capabilities will be to link the source of these visual params to one another. This way two different visuals could have some parameter that is shared in a way which will aid in creating visuals that are synched with one another.
This week - the project has some new developments in two different areas. The first is in the interactions between the touch screen interface and a visual that is being manipulated. In this example I show four different touch controls working in tandem to control a simplistic Fly Swarm visual plug-in. The graphics in this case are very simplistic - the more important aspect of this test is the interaction that is accomplished. The four elements that make this work are:
Enable Toggle: - used to completely enable/disable the visual
New Fly Trigger: - used to create a new fly with the current slider settings
Size Slider: - used to specify a ceiling for the fly size
Color Picker: - used to specify the fly's body color
Each of these controls can be used simultaneously / independently from one another.
I am pleased with these results - but the interaction needs to be, linked to a visual with more sophistication.
The second portion of the video looks at an investigation that I have done to modify an existing generative fish visual written in the same technology as my project: koi fish pond This processing "sketch" example seemed to fit my needs well. With some modification - I can use this example and apply it to my interactive visual plug-in - just like the fly example. Before beginning this process, I began experimenting with changing the graphics of the fish. For my visual - I prefer fish swimming side to side, as opposed to a top-down view. I create a series of new fish "skins" using open source images of fish from a textbook. This seemed to work nicely.
Second - I took this a little further and separated a layer to allow the fins, lips, spots, etc of the fish to be tinted a separate color from the main body of the fish. Now in the sketch I adapted the way the fish are painted by including a desaturated layer for the body and the fins layer. I select two random colors for each fish which is applied as a tint to the desaturated layers, and painted body first, fins layer second.
You get some funky color combos with a complete rgb random, but in the plug-in application of this visual, the user will be able to select more fine tuned color ranges for these elements using a few color picker touch controls on the surface.
Next steps, of course, are to do this integration work, and also add constraints on some other parameters of the visual - such as a central point for the fish to swim around, the ability to select a background image or another generative element, etc.
The lighting solution for the surface, as I mentioned, is a two-fold approach using the IR Led strip from 3m. The first strip of LEDs spans two sides of the plexiglass surface for direct illumination. This helps the camera see very clear blobs for finger touching of the surface as direct contact from anything rubbery (like a finger) will break the IR beam.
The second piece is the softbox illuminator for the ReacTivision tokens/fidicuals. This was better achieve with IR light as opposed to visible light as well. One major change included properly mounting the IR Lighting rig to the inside of the Softbox, and making the power cable for this component detachable by including a plug between it and the side LED strips.
- Software -
Still working out the ability to select Parameter nodes of a visual using a touch selection that will in turn populate a Touch Grid area with the appropriate touch control for that type of parameter. I have successfully worked out touch controls for a Single Value Slider, which can be reused as a subcomponent of things like ColorPicker, Sound EQ sliders, and other Value Sliders. This is a pretty basic component that has come along in terms of functionality since last week.
Of course as soon as I finished creating this piece last night some other unknown issue surfaced that will need to be worked out before I can proceed. However, at this point I believe I am closer than ever to connecting the dots between the touchable menu components and controlling some sort of example visual running in a separate window. Even if I just get the ColorPicker and a basic value slider working from the touch menu I will be able to achieve some interaction between these things for next week.
The next steps following this include linking the interaction of each parameter, not just to it's own native touch area, but also node-to-node and/or linked to a midi control. A lot of the menu and data model leg work to make these interactions were taken in to consideration all along the way - it will just be a matter of populating the components with the know-how to achieve these interactions.
Posts
