The first idea was (of course) the most obvious to try to make the video immersive by placing the viewer at the centre. To use a non-360°-videos for 360° video skyboxes does work but it causes for stretching of the video and subsequently damages quality and makes it hard to see details.

But it DOES work. So I can try to use more 'moody' 360 videos as backdrop and the more detailed content rich videos will play on other surfaces on top of the skybox?

Or build similar scenarios in Cinema4d with particles and put 360camera in the middle
or stitch together videos?

Ultimately I decided that 360° video distorts the video too much and diminishes the video itself - 1 main aim for this project is to keep the integrity of each piece of media I use intact.
Video Player tutorial
Using textures and shapes and blurry flashes of memory or association in one stream of consciousness


narrated by fragments of what people have actually said about the treatment ?
The soundscape that sound like a vast nothingness with an IV dropping droplets.
Projection from Flat 2D video to a Sphere projection that I could use as a hemispherical skybox?

Or travel into it in VR ; transition from outside into the molecule. 
What is the best method; to create a fully animated video and bring in into Unity VR or to animate rotating spheres and bring them into Unity and determine their flying patterns there?

Answer: create rotating models; animate further inside Unity
I am going to just focus on 2 main Phases:

1) Chemical molecular

2) Perception in the mind *

* This might include 'Out of Body experience' at some point into the borderless perceptual hallucination


The Ketamin experience is characterised by 'breaking open connections in order to create new ones'. Hence, the transitions are a focal point and driving element in the journey. These transitions also offer the ideal framework to bring together diverse media, like picture/image, video, 3D model and audio - which is the key defining element to the 'Video essay' genre.

Materiality and Shapes will also play a key role.
I see my role as a digital curator who's task it is to finds a new media format and narrative and storytelling method for selected artworks into an immersive VR environment.
Nudibranch are small, colourful sea slugs, they are soft and squishy and their colours are exciting and enticing - because they seem like such friendly creatures somehow the idea of the Nudibranch as a spirit guide in out of body experience came up. The idea also inspired by games like Zelda, where Link has a small fairy that guides and accompanies him or like Peter Pan, who has Tinkerbell- it seems like a comforting presence to have someone floating by you and guiding you.



> SHADERGRAPH! > in one of the Unity Tutorials with Luca we looked at Shadergraph and created a shining, floating blob - this seemed perfect. However, I needed to change Unity to the High Definition render pipeline. By doing this I lost the textures on most of my models. Even on the blob itself that I had taken from the tutorial session (created with the Node system). To relink these material I only found a entry on Unity answer that suggested I would have to understand a lot of the system behind the effect to manually relink my textures because the 'automatic relink to HD Renderpipeline' didn't work for me as it 'should.' Hence, I gave up on this path. *also because when trying to set-up an entire project from the get-go as HD RP, Unity warned I would be likely to run into bugs and I feared that I might run into even more difficulties when trying to test/run it in the DBL.
Seeing the same scene from multiple perspectives; leaving your body:
Sounddesign is going to be a huge aspect to bring this together!!!!!

1) Idea to add structure : Voice from the off - an essayist - giving the context to what is seen

2) Sound that includes spoken sound bits
Using video as texture tutorial
1) I picked up on the round shape of the molecules by projecting them on a sphere in C4D


2) I tried playing a video on a plane in C4D and applied Cloth simulation on the plane to ad some dynamic to the flat video. However this effect is boring and devalues the media itself.




I'm noticing that everything I am gravitating towards is very tactile and haptic , also softness , billowing movement is really fitting - all things soft and flowy and hazy.
Height map - turn picture into landscape


If I want to have a video play on such an unusual surface I think I need to UV unwrap it first..

Could do this with a mesh from a room photogrammetry and project something else on it and rotate it and put the camera inside..
Immersive animation set-up. The viewer/player is sitting in a room but standing in a virtual scene as the action unfolds around them. This is done by creating a timeline that tell a story through animation and guided timely transitions between different scenes.
Could create photogrammetry scan of person lying or sitting down.
What are interesting transitions? Playing with dimensionality?
e.g. Maniac when Owen jumps out the window and lands in the toy world.

Things morphing into each other!
* Edit: My initial idea/hope was to have different media types like 3d models, video and picture have fluid transitions that allowed the viewer to feel like they flow together. Turns out this is not possible the way I had imagined.
Walking in a landscape with different videos playing. I feel like walking or existing in a SPACE - giving some kind of HOLD for the user. Not catapulting the viewer into a digital abyss.
Could optionally transition into a model that has projection on it.

e.g. have the model be ON the Videoprojection plane


On that mesh could be for instance a reflective ball and inside that ball could be the coralline texture of Ketamin.
Noise - particle - node - texture - Shader graph
This light can turn into the nudibranch - spirit guide - that becomes a shape shifter - using the footage from the Mind Game anime.
First going into the dark space - where molecules fly around you like planets.

Transition: fly into planet - place goes white or other way to switch into new scene.

Maybe not standing around just in the dark but a very low lit space..
Visual inventory - visual chains of events - focus on transitions between media
It's an interesting approach to try to externalise objects from the videos. Make them tangible. Make them experiential.
This commercial/advertising video for COMME DES GARCONS PARFUMS is a very conceptual own form of 'video essay' because it shows the experience of the scent (olfactory) through visuals - alternating notes of the fragrant with images that are extremely focused on colour, shape, texture, haptic triggers that connect together thanks to their visual similarity. At the same time the video plays with the idea of memory as scent is known to trigger memories strongly and the voice over from the video gives a new context that connects the visuals further and again reminds of an abstract video essay. What I especially like and what I want to keep/ translate to the VR experience from the video is the way in which it interjects memory fragments and uses a voice over.
*What if the video gets projected on this and it THEN turns into a Shadergraph Nudibranch after the projection?

>> Edit: too difficult to implement currently
Leaving the room could also be a kind of sinking away of the space..might be too much effort though..
Extremely striking about Heyman's work is that his render quality is extremely tactile. The models feel at the same time real and touch-able but have a plastic 'Modelbau' quality about them.
The question remains; What program does he use and How could I get this kind of render quality????

Too adopt exactly this quality I briefly considered to take this image and use it in a Billboard effect inside Unity but it is of course not at all the same. Instead I looked for similar models.
I want these kind of organic blobby shapes with the colour combinations blobbing around in thespace!
My first attempt to translate this video to VR was to check and see if I could extend the impression of the all white space (a defining feature of the video) simply by playing the video in an all white skybox. This worked for parts, where the white hues matched ( see image (1) ) but didn't work in parts where the white hue didn't match. ( see image (2))



Also I wanted to adopt the reflective floor (see image (2)) but soon learned that these effects work in a 3dmodeling program like C4D but are generally too processing intensive for Unity - or rather if I want to create such an effect I need a lot more in depth knowledge of Shaders and Unity etc.
IV monitor screen - connect it to main camera so its always at bottom of screen
From here the boat could turn and the environment goes black.

Isen't this what happens in the Imaginarium of Dr. Parnassus?
Still I'd like everything to be a bit more soft & hazy.
Intention: For this Video essay task and I want to return more to the core thread of the video essay genre and work out new ways of STORYTELLING IN VR with VIDEO ESSAY aesthetics.
Just this one Image with the under titles immediately conjures up the feeling of a story to be told - also giving film noir + David Lynch feeling.
By collaging imagery you can also create connections and transitions

e.g. this texture could turn into a terrain.
Big improvement; by reducing the scene to more of the essential elements the message comes across much clearer and it's not visually chaotic and overpowering but just interesting and weird enough.
The cube now resembles a television you might see in a dentist office and gives a focal point when entering the VR.

Might be nice to have it slowly appear - like grow from a very small cube into the size seen here.

I also like the idea of having a chain of cubes that extends into the sky - maybe with the hinge tool? - and to simulate the out of
body experience the camera could fly up and ultimately see the white dot grow smaller and smaller giving way to a black space until it finally disappears and vanish into a completely black space.
This could be on a moving soft curved plane, with video cube in front of it floating in a starry galaxy that slowly materialises..
Objects seen in total black surrounding. - potentially dark terrain floor.
Floating fonts in the air.
Create sculpted shapes through photogrammetry scans!
Conclusion: Total blackness does look like the framework that most effectively showcases the artworks - BUT TEST IN VR.
Atmosphere - soft - dreamy - haze
Concept:
Technical approach:
To create an Immersive Essay about Ketamin therapy. A new kind of drug therapy used to treat treatment resistant depression. Due to the Stigmatisation of Ketamin as a recreational party Drug the topic not only deals with issues mental health but also societal prejudice and the rift that is perceived between medical and recreational drug use.

(How are they different?)

The objective is to use Virtual Reality as a medium to to show a bodily and biochemical journey of perception that patients travel through without leaving the room during their Ketamin infusion.
Design approach:
Ask for help to project video on surfaces?

Answer: NEEDS TO BE UV UNWRAPPED - THEN ADD VIDEO PLAYER INSIDE UNITY

OR TRY PROJECTING

Add projector - assign material > now would need to write Shader - work around might be to use movie texture
I struggled a great deal with the terrain because I tried several different terrain models downloaded from Sketchfab but could never get the same exact look.

Probably I should have experimented more with the actual terrain feature inside Unity, but it was not very intuitive hence I didn't get too far with it without investing more effort.
Mentioned during one class session:
possibility of photogrammetry model made of youtube video? - photogrammtery software that can take video and create models from it?
Claire Hentschker


Shahab Nedaei:
Disarray, 2018
> Interesting example of video surface projections
> might be created with a particle system and videos inserted?

What kind of display even makes sense it this experience??
Motext on spline tutorial used for creating 'trains of thought poem'
Adding texture to the IV poles because the whole surrounding is very white! 
> check to see how this looks in VR?
Moved the trajectory of screens out to the side to allow them to turn into a type of skyscape..
Maybe add rotation to the skycubes?
I really like this form of presenting 'All you can feel' by Sarah Schönfeld the video rendition from the digital exhibition, because it takes the video and warps it from a flat surface to a sphere, which is more reminiscent of the molecular shape both seen in the photo prints but also the chemical molecular structure itself.


It would be better to look for a surface material that fits better contextually and not repeat the same material in different parts of the scene..
However, this is the only material that maps nicely onto the texture without showing the pixelated edges.

Edit: one of the reasons I later abandon these shapes is because they are not fully suitable for using different textures on them or playing different videos; they create a pixelated look.
Both visually interesting and a good 'fix' is to have an object obscure the point where the video flows together unnaturally on the projected sphere.. Gives the video an effect it would otherwise not have without showing the technicality behind it.
Set entire scene in galaxy? 
Example of how the surrounding can be nothing and still something
Atmosphere is created by creating a surroundings, a space for things to exist in.
Idea: experience can become individual by the scenes the viewer sees- maybe only a selection? Maybe the course of scenes can differ?
From Paul I earned that anything involving reflection or shiny-ness is extremely difficult for Unity to compute so all of these things should be Baked in Cinema4D beforehand!
This has no reflectance - no shiny-ness
>
Next question: How do I bake it properly so that it keeps the shiny-ness?
Everything in one scene and create transitions within it

create sphere in cinema - invert normals

flip normals shader also for Unity

post processing blumen effect
Thoughts during process:
Think about how it works when inside - because moving people around in VR causes motion sickness and maybe it's also more clear that everything happens in your head when you aren't moving by having things appear and float by in a scene while you remain seated...maybe from time to time video plays on the seat?
Maybe the seat should be in all scenes?
*maybe instead the transition is a sinking so that you don't float by being lifted into the air but the entire floor moves up and the fragmented projection surfaces are lifted to appear piece by piece around you.
*can these cubes disappear by pop-ing? 
*use this softy-texture on walls, with more or less opacity
*still a thought to wake up in a 360 klinik room
Inside the textured sphere - another sphere with reversed normals - and inside is a galaxy with this set-up and visual inventory
From there when you exit the orb everything has turned to black for this scene and then with the timeline light reveals a new landscape.

Or reverse - you exit into landscape which then goes dark as sun sets to reveal the most abstract scene in black and then flash to wake-up


and end with glowing shader made with nodes!
before anasomnia plays in a patched together 360°video and then END!
Flowers disappear by deflating ? + with sound (slurp?)
and IV poles grow humongous.
COURSE OF EVENTS - TRANSITION:

Enter in clinic typ room - mind game video plays - the cubes pop( or shrink until invisible) and the chair sinks slowly while the warped surfaces appear out of the ground.
The molecules play on balls; then the underwater vibe with nudibranches as colourful collage - then going into one of the orbs - inside is the galaxy. When exiting the galaxy the clinic elements have disappeared and we are in a warmly/sepia skybox in the evening glow - the sun sets and we enter total black space where abstract videos play - they fade slowly to leave only a glowing blob made with shadergraph. It comes closer- then suddenly white and video of anasomnia all around - END.
All animated with timeline / defining when what happens
I need more ATMOSPHERE - solid ground + everything happens from the chair
Imagine this object set - up in the snowy landscape!
Going into this texture we could go into the girl sphere
Does it work better with scenery?
Test in VR!


Answer - YES!
Insight from VR view
- add the rain texture to the bottom to make it feel like there is water below - somehow weather feels like it plays a role here

>> immersion idea: can I add air z.B to the experience - time it with a ventilator ?

reduce audio to one audio player to kill the echo
figure out how videos end...
transition between scenes with timeline? * bring Box into haze scene ?
make plants sway?

how to connect black space and haze scene?
* add Ana Roxanne to black scene - potentially find more videos for this scene or create another sculptural instillation piece for the slumber video ..
Reddit comment to script - objects to sway: https://www.reddit.com/r/Unity3D/comments/9nxil2/script_to_make_object_sway_gently_from_left_to/ e.g. flowers + plants
The idea was to adopt this style by building up one industrial looking screen and enshrouding it by nature.
This approach quickly became messy.




Frequently in the process I would pile on more and more elements. realise they don't work in unison and move on with just one element and reducing the scene again. Sort of an 'add and subtract flow'.
When looking at this scene from a VR project i really admire by . I am surprised because this approach of 'simply' making isolated objects float in the air would be something I couldn't accept if I did it but I like that he did it... The floating plane seen here let me think 'ok, maybe I can use simple plane as a video projection surface and just place it in my screen.'
Once I now have all the elements in place I have to work on transitions and connecting the scenes.

* add physics like a floating and bopping and bumping into each other onto the nudibranch blobs
* can also be animated with timeline ( but unnecessarily difficult?)

* edit video differently since it will be a flat layer put in front of the users eyesight - meaning editing becomes more important because it's the center of the screen * cut ana - add björk landscape

* also possibility to add typography layer - e.g. untertitle of the poem in the more black scene or untertitle for Ana Roxanne

* what can shaders do in terms of lighting and shine or realistic look?

* make the scenes fade into eachother
The music of Ana Roxanne beautifully captures a calming, ambient mood - something I imagine would be played during a Ketamin infusion to calm the patient and guide their experience into a calming direction in order to prevent having a 'bad trip' or fear someone might have when experiencing a K-hole. The tracks I choose (1) immortality and (2) slowness are especially fitting because they have the added element of spoken words, that seem like an inner monologue.

The 'Spiritual Combat 9-A with mo chuisle' also has a rather strange mix of ambient sound and spoken narrative along with sample sounds that seem like a strange trip in someones mind - remembering fragments.
Source for inspiration of VR formats: RadianceVR : https://www.radiancevr.co/
Found Media that I want to integrate into the VR experience - link by association to Ketamin:
Conclusion/evaluation:
I certainly got to understand the basics of the Unity engine as an EDITOR! Coming from working almost exclusively with the Adobe suite - where each program is set up in the way that you work mainly there it took me a good deal of time to fully understand that Unity is the engine you throw everything in and only assemble. Seems like a very basic, easy to get concept but took some time to fully understand the limitations and possibilities of the program.
Attempts at translating the Nudibranch into VR (1)
Sucess at translating the Nudibranch into VR (2)
I was also super inspired by this project of CROSSLUCID created for Spectacles AR lenses that allows you to see these organic textured blobs floating around. I was super fascinated by these shapes that in a way reminded me of an abstract Nudibranch.

1) I played around with the particle system inside Unity because I thought that maybe I could create a sphere with a trial behind it playing a video that could create a similar looking, living, growing, blob. However, I couldn't get that to work - in part maybe because my computer doesn't have the strongest processor so I didn't feel comfortable trying out settings for a long time.


2) Instead I aimed to create these shapes in Cinema4d. First I tried using physics(wind) on Softbodies, which didn't work.
Then I found this
Tutorial that allowed me to use 'Metaballs' in C4D to create nudirbanch shapes
(https://www.youtube.com/watch?v=gyduDSS6TZg)
Process trials/scenes:
The process of creating these shapes and getting them into Unity included:
Metaballs > Turn to OBJ > put OBJ into Instant Mesh to reduce mesh > Shape in C4d > fix mesh holes in C4D > apply texture of Nudibranch image

PROBLEM when importing into UNITY > textures /material wont relink and map properly > work around solution finally found by > BAKE OBJECT inside C4D > then import into Unity

> Still would like to learn : HOW CAN I BRING THE SHINY SURFACE LOOK INTO UNITY? (Doesn't work/translate with Bake object or bake texture)

> FINAL STEP: MAKE THEM FLOAT AND BOB AROUND - using physics inside Unity?
Attempts at translating the Video into VR (1) 360°video
Another idea I tried was to isolate each video snippet and play them all on separate cubes that together formed one large mosaic and have those cubes turning - perhaps on each side of the cube a different video could play so it functions as a mosaic-cube-puzzle? (the wall could additionally have code on it that would allow the user to push the cubes - or potentially move the cubes with the controller - but I am not really interested in making the experience interactive.)


My first approach for the entire project was to try to create groups across different video that connect visually, which I also tried to implement in connection to the above mentioned idea:
2D Image
Video
Terrain/3d space
Terrains inside Unity
I experimented a lot with these sculpted almost cloth like fragments to display image and video material.
Initially I imagined these fragments to fly past the viewer/ the viewer to fly through the fragments but considering the high probability of woozy-ness and nausea probably induced by this approach that was a bad idea.
Only a few of the image textures would map nicely onto these surfaces and particularly when playing videos on the surfaces via a Videoplayer it became clear that if I want a smooth and controlled mapping of the video that wouldn't appear pixelated I would have to UV unwrap them in a time consuming effort that didn't same worth it.



Ultimately from these trials/try-out with fragment came the realisation that I really do need a space to place my actions in otherwise the viewer is completely lost.
My first experiments are to change the physical shape of the media into something more three dimensional
Rather I should try to create interesting shapes that can overlap intersect with the field of vision of the user and create some visual excitement and also new angles to see the video material from.

Aiming for an effect like this: Leaving the viewer flying through different morphed planes that might not contain video but images that float by like fragments:

These shapes could be VISUALISING THOUGHTS THAT ARE NOT FULLY FORMED AND CANNOT BE GRASPED AS THE FLOAT BY! - A strong analogy for a Ketamin experience.

Realisation/Actualisation ideas:
Here, briefly, the video texture and object texture work well together - but this isen't true for the entire video sequence.
Video mapping
Difficulty:
This is the first video I knew I wanted to integrate - it is from my favoruite anime 'Mind Game' and the scene takes place in a white space, the character is disoriented and wonders where he is while the other figure keeps changing form/shape shifting because he can't decide what the figure should look like. I feel this is a strong representation of the disassociation and 'breaking upon connections to form new ones' said of ketamin therapy trips
Specific Actualisation idea:
First Scene
Transition (1)
Media used
Final Scene
Closing Sequence
'THE SCENT OF A FLOWER WE HAVE NOT FOUND'
Second Scene
Integrate black scene Scene into this scene so that their is only one switch from dark to light
> edit Video into two pieces
> * possibility to add flower? 
> play smaller separate video walls in between
> find out how to create effect of walls dropping away like sides of a box to reveal the second scenerie > transition into next scene


> Additional idea for transition: use the screen in field of vision to play the flowers at the end of the Waterfall + Tuch sequence
To-do ( finishing touches)
Realisation about work process
Trials and Dislikes
Clinic space inspritaion
For this show(GUCCI F/W 2018) a handful of elements where used to create a hyper-stylised and chic portrayal of a clinical space. The association is immediately clear without being 'too much' or too literal.

Inspired by this I added a blanket over the dentist chair to make the model fit better into my scene and add a stylistic element on top.
Idea of the spirit-guide in an Out-of-Body-Experience
Videos that focus on shape and material
Transition (2)
Is the most elegant way to fade everything to white by teleporting out of the scene?
> Tutorial suggestion to just create a white face at both the beginning and end of a Scene >https://www.youtube.com/watch?v=CE9VOZivb3I
Decision to change the project title to something more poetic and metaphorical, which is hard to make sense of but conjures up a mental image: from A twilight slumber' to ' the scent of a flower we have not found'

With this eluding to the flower I want to integrate video footage of flowers throughout the experience (they are already in my scene)

Additionally the title eludes to another sense (sense of smell), which in turn could become more important an immersive in an instillation set-up
Font try (1)
Font try (2)
Font try (3)
To Bring the reflection into Unity!
Media used
"Extracted and Circulated is an ongoing collaborative platform created together with Sasha Litvintseva and Graeme Arnfield, surrounding moving image as compilation, as assembly and as the production of new work. The programs commingle passages from our own film work with images collected and re-presented from the disorganized archives of YouTube. Allowing each video to become starting points for each other, the project aims to dissolve the hierarchies of found, processed and produced material. Accompanying the screenings we speak about issues surrounding authorship, the value of images, the provenance of their intentionality, their affect and materiality."
EDITING style heavily inspired by Extracted and Circulated project
Bringing everything together:
Floating objects script
> I would like to use a physics script for this as I believe it is one of the benefits of Unity when looking for random movement of a lot of objects to apply a script instead of set all the keyframes for an animation


On unity answers I found a code that allows me to adjust singular values through script
-https://answers.unity.com/questions/59934/how-to-an-object-floating-up-and-down.html

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class SimpleFloating : MonoBehaviour {

public bool animPos = true;
public Vector3 posAmplitude = Vector3.one;
public Vector3 posSpeed = Vector3.one;

public bool animRot = true;
public Vector3 rotAmplitude = Vector3.one*20;
public Vector3 rotSpeed = Vector3.one;

public bool animScale = false;
public Vector3 scaleAmplitude = Vector3.one*0.1f;
public Vector3 scaleSpeed = Vector3.one;

private Vector3 origPos;
private Vector3 origRot;
private Vector3 origScale;

private float startAnimOffset = 0;


/**
* Awake
*/
void Awake() {
origPos = transform.position;
origRot = transform.eulerAngles;
origScale = transform.localScale;
startAnimOffset = Random.Range(0f, 540f); // so that the xyz anims are already offset from each other since the start
}

/**
* Update
*/
void Update() {
/* position */
if(animPos) {
Vector3 pos;
pos.x = origPos.x + posAmplitude.x*Mathf.Sin(posSpeed.x*Time.time + startAnimOffset);
pos.y = origPos.y + posAmplitude.y*Mathf.Sin(posSpeed.y*Time.time + startAnimOffset);
pos.z = origPos.z + posAmplitude.z*Mathf.Sin(posSpeed.z*Time.time + startAnimOffset);
transform.position = pos;
}

/* rotation */
if(animRot) {
Vector3 rot;
rot.x = origRot.x + rotAmplitude.x*Mathf.Sin(rotSpeed.x*Time.time + startAnimOffset);
rot.y = origRot.y + rotAmplitude.y*Mathf.Sin(rotSpeed.y*Time.time + startAnimOffset);
rot.z = origRot.z + rotAmplitude.z*Mathf.Sin(rotSpeed.z*Time.time + startAnimOffset);
transform.eulerAngles = rot;
}

/* scale */
if(animScale) {
Vector3 scale;
scale.x = origScale.x * (1+scaleAmplitude.x*Mathf.Sin(scaleSpeed.x*Time.time + startAnimOffset));
scale.y = origScale.y * (1+scaleAmplitude.y*Mathf.Sin(scaleSpeed.y*Time.time + startAnimOffset));
scale.z = origScale.z * (1+scaleAmplitude.z*Mathf.Sin(scaleSpeed.z*Time.time + startAnimOffset));
transform.localScale = scale;
}
}
}
Brackey tutorial for scene tranitions: https://www.youtube.com/watch?v=CE9VOZivb3I&t=939s
Brackey tutorial for timeline use:
I continue to have trouble with material re-mapping correctly in Unity

> the Nudibranch - does the image get more pixle-ly when baked as a texture ??
> Fix coloursheme coheseiveness
The scent of a flower we have not found
By attaching the screen to the users field of vision a virtual 3D environment AND the 2d plane typical for video are brought together. After a few experiments trying to bring video on to organic curvy shapes this proved (1) time intensive because a proper UV map would have to be created for the video to map in a controlled and non-distorted way (2) most probably the 'added' positive effect of giving the video a 3d dimensionality wouldn't be adequate the effort because video information might be lost to the viewer and the integrity of the video should remain intact.
As an added effect each users experience will be unique as they are free to move their head and depending on the users movement video and surrounding will match/look unique to their perspective.
Audio in Scene
Ana Roxanne - Immortality
length : 4:04
spoken words
I was only dreaming - a past encounter resurfacing

I was only dreaming a familiar gesture is placed in time
I was only dreaming of something I left behind
Things that vibrate together
thing inmost nature seek on another
Inspiration- add text in Image!
Idea (1) give subtitles to what Ana Roxanne is saying

(2) try to add excerpts from Shannon Powell poem
This script also worked but didn't achieve exactly desired effect:

using UnityEngine;
using System.Collections;

public class Floater : MonoBehaviour{

public float FloatStrenght;
public float RandomRotationStrenght;


void Update () {
transform.rigidbody.AddForce(Vector3.up *FloatStrenght);
transform.Rotate(RandomRotationStrenght,RandomRotationStrenght,RandomRotationStrenght);
}

}
This script for floating would have had the exact effect I wanted but is no longer on the assset store :https://www.youtube.com/watch?v=a2wpVFjY0lM
Course of events

(1) Intro video 1 plays on Awake on the 5 planes that form a box
(2) Video stops then plane on first person controller plays
(2.1) at the same time music starts
* still pending decision on what fabric is on the chair
(3) waterfall and tuch play
(4) videos stop playing and two planes disappear
4.1) music transitions to video on cube planes again
(5) Video on cube box plays again
(6) video stops and planes collapse outward
(6.1) ceiling plane flies away; all planes move simultaneously

ENTER NEW SCENE  > Idea: maybe trigger new scene by having a box collider as trigger move toward chair ?

(7) new scene loads
(8) cubes with video player expand until visible end-position
(9) Mind game video plays
(10) Video ends and cube shrinks until invisible

TRANSITION TO FINAL SCENE > either load new scene or make objects slowly appear and grow from the ground..
(11) have edited video play on plane in first person controller view
(12) video ends - plane disappears but music continues playing
(13) objects start disappearing by sinking into the floor
(14) the title appears as everything fades to white
What EXACTLY do I have to animate with the timeline feature?
Maybe I need to export and insert the audio separately because otherwise the audio will end once the video stops playing and it won't be a smooth transition.
Transition To-do
* Note: I like the bubbles of the nudibranch models so if I still add some they should have the bubbles

(1) done
(2) done
(2.1) done
* still pending decision on what fabric is on the chair
(3) done
(3) done
(4) done
4.1) done
(5) done
(6) video stops and planes collapse outward
(6.1) ceiling plane flies away; all planes move simultaneously

ENTER NEW SCENE  > Idea: maybe trigger new scene by having a box collider as trigger move toward chair ? or transition with script?

(7) new scene loads
(8) cubes with video player expand until visible end-position
(9) DONE
(10) DONE

TRANSITION TO FINAL SCENE > either load new scene or make objects slowly appear and grow from the ground..
(11) have edited video play on plane in first person controller view
(12) video ends - plane disappears but music continues playing
(13) objects start disappearing by sinking into the floor
(14) the title appears as everything fades to white
To make an object appear and disappear at a desired time in my timeline i need to use ACTIVATION TRACKS - this tutorial explains that : https://www.youtube.com/watch?v=60Py1UqUJQo

* do math on the waterfall so it disappears as it is retreated
For future - how to create uniformity in sclae and position across multiple scenes?
The flying poem will only be really interesting once it is animated so that the words move along the spline ( for this I would probably have to create a spline in Unity and couldn't use the OBJ. I created?)

+ The user can move around and walk through it; Idea of a walkthrough poem - depending on the words you walk through you can not only read them around you but hear them as you move around and closer to them.
Work around to make the planes rotate : https://www.youtube.com/watch?v=itIh8PYGP7w

Script is used to create a custom pivot through which the point of rotation can be changed - however do I still need this transition? Or could such a transition only work if EVERYTHING happens in one scene and there is no scene change? Or could it start tipping as the transition starts? - Problem is that for this scene the skybox is black and I can't change it to the skybox of the second scene without altering the lighting..
Experiments with space
My plan moving forward was to translate the snowy landscape from a project by Isaac Norton called True North, because it is a surreal feeling landscape that left a lasting impression on me.
As I second step I would try to place some of the visual collages I had created into this environment.
Inspiration
Insight
Visual collage experiments
Result
Decision
Insight
When viewing these scene in VR two things became clear

(1) To create an immersive environment you need a SPACE not just a bunch of objects floating around in a white skybox

(2) The Ice-landscape feels disconnected from the topic as well as from my style as a creative

I couldn't get these models to feel anything other than like 'free downloads off the internet'
Problem
How can I make the assets that I am using feel more like 'ME'?
Solution approach
Inspired by the work of Frederik Heyman I wanted to connect
the medical models that I want to use because of their clear connection to the topic of the Ketamin-infusion-therapy with nature. In his work Heyman uses many technical looking assets to create his visuals - the visuals for which he connects nature and these industrial elements inspired me for this project.
Large screen placement inspiration
Final step: creating the transitions between scenes
Tutorial 1 - by Brackeys creates a crossfade but it is as an overlay in a 'Flat scene'/Screen space only - I've changed the space to world space so that it it would apply to the 3D VR environment.
Still I need to adjust multiple parts of the code

* the if(Input. -) method has to be something other then GetMouseButtonDown because I want the scene to load on it's own; not that the user has to push any button; something that wouldn't work inside the VR experience anyways.



Also the IEnumerator is triggered by the "Start" animation of the crossfade animation sequence that connects two scenes through the LevelLoaderscript - however I think this needs to be connected to my Timeline animation but I am not sure how I can script this trigger connection differently.
Tutorial link
Other Tutorial link that uses Menu and Buttons to change scene in VR set-up
For the documentation of the project the switching of the levels might not be absolutely crucial
Audio Inspiration
In the context of the Ketamin therapy such string of 'random' videos can be understood as memory fragments.
Some tutorials used
Through during process
Inspiration: transitioning out of a scene
Project video on morphing, changing surface
Further To-do
Downloaded Models from Sketchfab used for the Unity scenes
To combine the industrial, medical equipment with the flowers into formations that have a sculpture like quality - inspired by Frederik Heyman.
Plan
Tipps to remember: it's possible to change the Image resolution size
Attempts at creating terrain inside Unity
Or solution approach: add trigger?
I know I have to use

UnityEngine.SceneManagement;

I don't know what my if condition should be to transition the scene?

Could I use a collider that I animate at the end of the scene? Then move the FPS controller into the box collider and create my if - statement out of that ..?


I should try to combine this simple script that uses the Box collider as a trigger ( which I will animate at the end of my scene) and the crossfade transition
Closing with the Title typography!
Big challenge:
Possible solution
From what I think I've understood I would have to UV unwrap each letter - to then map the reflection with a 'Bake texture' tag to then re-link properly inside Unity ...

Maybe it's an idea to just try a different texture on the font so save this stretch of work ..

Other solution could be to create the reflection inside Unity with a reflection probe!
Bake reflection execution problem:
Currently when I try to work with the mesh the font goes back to it's shape prior to being sculpted by me.
A work around might be to export the sculpted font as OBJ. and re- introduce it to C4D.

Then I might be able to UV unwarp it so that the reflection could be properly baked.




Whenever something seems like a large effort, with non instantaneous results I think 'this must be wrong, there must be another way'
Self-evaluation / insight :
THE SOLUTION!
Finally had the idea to re-create the way the reflection is created in C4D in Unity - did this with Reflection Probes and downloaded HDRi image.
Got the images from https://hdrihaven.com/ ( hdrmaps.com is also an option)

Tried out different types of Maps with different reflection qualities.

Steps:
Import downloaded HDR Map
> Change Texture Shape from 2D to Cube in Inspector
> Change Mapping to Lagitude- Logitude layout ( Cylindrical)
> Apply
so that you can insert the material into a Skybox / Cubemap

> Add Reflection Probe
> Under Lighting window turn off Auto Generate and click Generate Reflection probes from dropdown menu
Brackeys Tutorial on Reflection Probes: https://www.youtube.com/watch?v=lhELeLnynI8
Tutorial on HDR Skybox for Unity: https://www.youtube.com/watch?v=FrafK93FtX0
Tried this script: problem with the code in tutorial : I need to assign guiObject but cannot - same problem with the other variable used for this in his gitHub update - probably easy fix but showing up issues in my console

Have script OnTriggerLoadLevelV2.cs from his github: https://github.com/UnityTutorialsHD/Unity-Tutorial-Assets

I used a cube with the script on it - then disabled meshrenderer and used as box collider - the cube is then animated with animation track - on - off in the timeline

+ if I do the Gameobject - UI - Image and create a canvas with white image it just shows a black rectangle when I run the game in my view
Here, the scene changes if I use this code but I try to activate the box collider - used as trigger to only be active after 50 seconds but the first scene doesn't play and instead switches immediately to the second scene


> Move Scene script
trigger is cube with script on it
I have the levelLoader approach from this tutorial
Additional tutorial that explains Steam VR Load level script functions - their trigger is a laserpoint with the controller, which is not what I want ..so I'd need to adjust with boxcollider?
https://medium.com/@setzeus/steamvr-2-0-tutorial-scene-switch-2b2905591813
I've used the steam Vr loadlevel script on an empty gameobject but now it immediately switches to the second scene and the skybox of the second scene looks different