Week 7

The deliverables:

Compile a collection of movies that feature real urban sites. Focus on Rome, London, New York (and Sydney to a lesser degree if there is good material). Consider putting a call out to facebook friends to help, or searching for "Top 10 films set in London" or similar. 60 dvd quality films categorized.

Composite sites together between clips from the constituent films. You will have a show-reel for each specific location, to be used in presentations and for your reconstruction. Avoid video compression as this will negatively effect your reconstructions. Minimum 12 location show-reels.

(experiment with 20/10/5 frames per second to find optimal. Use Quicktime Pro or similar to export frames out.

reconstruct pointclouds, and attempt meshing using the procedure outlined for PhotoSynth toolkit (or alternative approaches). Minimum 4 meshed reconstructions.

Part 1
London
1.Sliding Doors
2. Snatch
3. Sweeney Todd
4. A Fish Called Wanda
5.The Other Boleyn Girl
6.Elizabeth
7.Casino Royale (1967) and new one?
8.Bridget Jones's Diary
9.Secrets & Lies
10.Gaslight
11.Shakespeare in Love
12.The Italian Job
13.Dial M for Murder
14.Bend It Like Beckham
15.The Ladykillers
Love, Actually
Notting Hill
About A Boy
Bridget Jones’s Diary
Closer
V for Vendetta
28 Days Later

Rome
1. The Talented Mr Ripley, Antony Minghella, 1999
2. Eat, Pray, Love (Eat part in Rome)
3. La Dolce Vita (Federico Fellini - 1960)
4. Three coins in the fountain (1950s)
5. Angels and Demons
6. Roman Holiday (1950S)
7. The Pink Panther 2
8. National Lampoon's European Vacation
9. Ocean's Twelve
10. Gladiator

New York
1. Sex and the City 2 (2010)
2. Wall Street: Money Never Sleeps (2010)
3. Date Night (2010)
4. 27 Dresses (2008)
5. P.S. I Love You (2007)
6. The Devil Wears Prada (2006)
7. The Interpreter (2005)
8. Spider-Man 2 (2004)
9. Shaft (2000)
10. 13 (2011)

Week 6

From my initial idea of using photogrammetry to recreate a scene, like Oxford street over a series of years. I have discovered that i need many more photos than what is available. This led to investigating film, tv and other media as a source for images. This idea could be used to inform design decisions in architecture, recreate scenes from an earlier point in time to be used and manipulated for film, or to document historical changes over time for council records or educational purposes.

At this point i am in the process of reconstructing high definition models a scene from a movie. This process raises a few issues like if you are extracting still images from film, how much of that content is actually real? Are you reconstructing a scene that is already a 3d model? I have reconstructed some test scenes to point clouds, ready to be taken to the next stage as 3d models. From here, these models could be used in applications such as google maps or a series of animations that could show changes over time, possibly combining real video footage and the recreated scene, this way it would be possible to blend past and the future elements.

Crows Nest Shop Reconstruction (point cloud)

Town Hall Reconstruction (Point cloud)

Church Reconstruction (Point cloud)

Finally a breakthrough! Here we have a model in MeshLab.

When it came to the dense reconstruction, with HD images, it stayed at the above point for 24 hours and didnt move. Im assuming it ran out of memory.

Precedent Study 6

Study 6 - ISPRS Journal of Photogrammetry and Remote Sensing - City model Enrichment - Philip D. Smart, Jonathan A. Quinn, Christopher B. Jones

This paper explores how freely available sources of georeferenced Web 2.0 information can be used for automated enrichment of 3D city models. There are many motivations for the production of these models, including:

visualising landscapes (planning and development)

virtual tours and games

radio communications network planning

providing detailed information about an individuals immediate environment through the use of Augmented Reality techniques.

Using these techniques, the information and geometry of nearby features can be overlaid on live video or still images of the environment. It is brought to my attention that the effectiveness of these 3D models depends not so much on visualisation but upon good quality annotation and information that can be associated with the model. There may be a need to link these 3D models to other sources for a richer experience, these may include Wikipedia where there is a large number of georeferenced articles about buildings, places, and other landmarks. OpenStreetMap and Geonames gazetteer are other resources.

http://www.citygml.org

"

The combination of the 2D and 3D models results in a well segmented set of buildings which are accurate to the ground plan, problems matching the two at times is a concern and methods to resolve this are being examined, for example using fuzzy matching techniques that take into account the proximity of multiple point references from the different sources and the identification of potentially important building geometry.

Precedent Study 5

Study 5 - Data Generation for CAAD with Digital Photogrammetry - Horst A. Breyer, Andre Streilein

This is a paper from 1991, however still valid in terms of fundamental ideas. I thought it would be worthwhile having a good look into it to see what the motivations were then and also what was available and where it was heading.

It seems that because of the technology available, there was a lot more thought put into which photographs to take and how many to take. The position of the cameras was all determined manually and lines were computed between points. The prcess showed potential at this point of time, and accuracy could be achieved with large well defined features. Manual and semi automatic measurements could already be achieved. Still it is unlikely today that it can be a fully automated process.

Precedent Study 4

Study 4 - An Update on Automatic 3D Building Reconstruction - Norbert Haala and Martin Kada

Photogrammetry has been around for around 2 decades. This paper looks at current approaches in order to comprehensively elaborate on the reconstruction methods and their principles from polyhedral building objects which represent the blueprint of a building and the roof to more detailed facade geometries from terrestrial data collection.

It seems to be that fully automated image understanding is hard to solve, semi automatic components are usually required to support the recognition of very complex buildings. Suitable image matching software (like what i am using -Photosynth), can generate 3d point clouds and at an accuracy, reliability and detail which was only feasible by LiDAR measurements.

The interactive visualisations of 3d city models were open to the public by applications such as Google Earth and Bing MAps. These visualisations are very coarse and limited to roof structures and planer facades. A considerable number of approaches have been developed to collect 3D building models at acceptable effort for complete city area, but such models feature flat facades and distinctive roof structures. This data is available to a large group of users (via Google Earth) and frequently maintained and used for planning purposes. Facade is however, the most identifying feature of a building usually and the problem is that this is emitted from these models or is very flat. I want to develop this in my project and perhaps think of ways that these facades can be recorded at greater detail.

"Applications like Google Street View or Microsoft Streetside use images collected from such mobile mapping systems to generate panoramic views of urban areas at street level. In this context, these views supplement the oblique images from airborne platforms. Georeferenced video streams as collected from mobile mapping systems can alternatively be used for more advanced real-time visualizations. As an example, large scale 3D reconstructions of street scenes are e.g. required for car navigation systems. Such an approach, which aims at the generation of realistically textured surface models at video frame rates, is e.g. described by Cornelis et al., 2008 " - N. Cornelis, B. Leibe, K. Cornelis and L. Van Gool, 3D urban scene modeling integrating recognition and reconstruction, International Journal of Computer Vision 78 (2–3) (2008), pp. 121–141. Full Text via CrossRef | View Record in Scopus | Cited By in Scopus (38)Cornelis et al. (2008).

http://www.sciencedirect.com.wwwproxy0.library.unsw.edu.au/science/article/pii/S0924271610000894

Google Earth 3D

Precedent Study 3

Study 3 - Digitally Documenting D-Day: The use of close range digital photogrammetry at Pointe Du Hoc - Richard Burt, Ozgur Gonen

Robert Warden, Vinod Srinivasan

this is a good case study of a project using photogrammetry to digitally document the command post at one of the most historical sites of the D-Day landings: Point Du Hoc. A website will be created for the project allowing an interactive tour. The methods used to collect survey data for both the production of 2d Historic American Building Survey drawings and for the 3D digital model are described.

3D laser scanning - ability to create very accurate 3D representations but the amount of raw data recorded can be unmanageable, especially for outdoor scenes.

Image based modelling techniques - inexpensive and can capture objects of any size. Some systems use computer visual techniques such as computational stereopsis to automatically determine the structure of a scene from multiple photos but are limited by the strength of the stereo algorithms used. Multiple tools/software may have to be used to achieve the desired results., this i am noticing more and more. They speak of a program called Facade which is a research prototype.

An important element to consider is lighting. If the photos are taken at different times of the day, different days, different times of the year, the colours and lighting will not match from photo to photo causing inconsistencies in the final output. These can be altered but slightly to match more but will take some time and skill to match the shadows and highlight. The dark areas make it difficult to identify corners etc ie in a dark room.

Obtaining 3D measurements from the images is done by marking the same point on the building on at least 3 images taken from different positions, and then when sufficient information is collected, computer software is able to locate theta 3D position of the cameras. Once the camera positions are located, a series of calculations locate the position of all marked points and create a 3D model. Below is an example of the workflow.

http://cumincad.scix.net/cgi-bin/works/Show?_id=sigradi2005_634&sort=DEFAULT&search=photogrammetry&hits=37

Precedent Study 2

Study 2 - An application of Photogrammetry Measuring Technology to Parametric Modeling of Korean Traditional Wooden Structure - Han-Seon Bae, Ma-rie Kim, Soo-Young Shin, Hyun-A Kim, Chae-Shin Yoon

This paper is about constructing a 3D mole of a Korean traditional wooden structure, with the purpose benign to obtain dimensions. This was done using PhotoModeler, which extracts information from a series of photos. All you need is one measurement to act as a constrain and then the other dimensions can be generated. The possibilities of this are numerous. I can imagine if you were developing a townhouse, you could measure the street length of one and have the program generate the surrounding buildings automatically. It would also be helpful in documentation processes and restoration projects.

Using PhotoModeler, like any photogrammetry software, relies heavily on the quality of photographs (resolution and position of cameras) in order to get good results.

http://cumincad.scix.net/cgi-bin/works/Show?_id=ac57&sort=DEFAULT&search=photogrammetry&hits=37

Precedent Study 1

Study 1 - From 2d Base Map to 3d City Model - Matti Arponen - City Survey Division, City of Helsinki, Finland

Laser-scanning and digital photogrammetry have been used in collecting 3d information on map objects in Helsinki. These methods have been applied in mapping out dated map features and in producing 3D models manually, semi-automatically, and automatically. It is noted that it is as much a software development project as it is a 3D city model project.

Some of the problems encountered in this effort (and some i can relate to) include:

finding quality data sufficient for 3D

how should this be connected to the bigger picture of the cities CAD base

Is there enough height information

how can the skills of 2D draftsmen translate to 3D

how is the model to be updated

what level of detail is to be modelled

In this case there were 3 models at 3 scales: a rough city model, a precise city model, and a textured precise city model.

The textured and precise city models have been produced by using a combination of facade photographs and manual and automatic modelling techniques.

The products used in this instance are TerraModeler, TerraScan and TerraPhoto, which re under constant development.

Gathering height information is a big project. Height modelling by an operator using stereo photogrammetric work or interpreting laser scanned points (ground level, buildings, edges of the street elements as break lines would take up to 25 years in this case. Laser scanning as a method to gathering info is being experimented. The problem with these methods are, unlike stereo mapping where only a few points are targeted and measured, the scan picks points which may be of no interest, like humans, traffic lights etc so it may take some time to use this information. The benefit of coarse is the level of detail.

http://cumincad.scix.net/cgi-bin/works/Show?_id=a35a&sort=DEFAULT&search=photogrammetry&hits=37

Week 5

This week i have focussed more on research and the precedent studies....mainly because my computer has been out of action. I have resolved a few issues that i have had over the past couple of weeks but technology has not been kind to me lately. Seeing how i cannot write much on progress.....below are my troubleshooting notes and research i have found helpful. The blog is turning out to be quite a good way of organising my ideas and progress.

notes:

make sure firewall settings on low

UAC off

try on uni lab computers

check the image files are .jpg not .JPG (if you have JGP.key files generated it will not work as it is expecting .jpg.key

"file not found" would seem the process hasn't worked to that point

the openGL error could be to do with the video card

try using different values in the Bundler process

try jumping to the next stage in process

SFMToolkit wont work either

no PMVS model generated

all synths can be viewed at:

http://photosynth.net/userprofilepage.aspx?user=lloydramsay&content=Synths

a good source of historical American movies http://www.archive.org/details/movies

Josh's reconstruction from video frames http://photosynth.net/view.aspx?cid=5fa1b8d1-7b6b-457b-a8c0-a2d44c4b4184

PhotSynth Toolkit tutorials http://vimeo.com/14796939

3d reconstruction using Bundler and PMVS http://www.vimeo.com/14783202

a really interesting reconstruction could be done of the streetscape/surrounding areas around a major landmark.

source images: http://subjectguides.library.unsw.edu.au/content.php?pid=28808&sid=210438

Choose a location which is changing a lot now and document that

Choose a tourist/landmark location that will have a lot of images; not necessarily in Australia.

3D printing: http://www.shapeways.com/

PhotogrammetryGuide - Josh http://files.neonascent.net/photogrammetry-guide.pdf

use meshlab to view .ply file @ http://meshlab.sourceforge.net

Week 4

This week i have spent hours and hours experimenting with ways to complete the PhotoSynth process using the PhotoSynth ToolKit 7. There seems to be a glitch with the toolkit that prevents the geometry being formed. Not a lot of progress on paper, but many frustrating hours in front of the computer.

I hope that when i can get the process happening smoothly, the project at the moment has a heavy focus on the process rather than the final product, i'm hoping this will turn around soon and i can start experimenting with some output.

It is interesting that different stages work differently on my 2 computers and some not at all. It has been another tedious week, tricky staying motivated when there are these technical issues which are hard to move past.

Week 3

This morning I met with Jeremy before the scheduled studio meeting to discuss general progress. It was noted that i will be updating my weekly reports when i get the project on track. It was also recommended that we each need to be spending 12-16 hours a week, i am sitting in the lower part of this range and that will be increased when i finalise the focus of my project.

Some areas i am looking at are:

Hyde Park Barracks

Oxford Street

Reservoir Paddington

I need to do a few tests of an object, a building and maybe a series of buildings.

also do a precedent study....something related to what you are doing. Critique it...what is good about it and what can be improved. What can be learnt, what i endeavour to do with it. It can be bad example and pinpoint how you would do it differently. The precedent studies should take up a third of your final report. Do 1 or more each week

I managed to pull together a few tests with Photosynth this week. I took some photos of a car showroom in artarmon and discovered some initial problems. This gave me a bit of direction and a starting block.

I met with Josh on Thursday for a tutorial on how to take the right images which was beneficial. We started off in town hall and then shortly got the boot so headed to a church on Castlereigh St. I then processed these using Photosynth and the results were pleasing.

After discussing developments with Josh, we have started looking into film as a source of still images to capture and compare areas of interest over the years. This is beneficial as a lot of images can be pulled from film. Yet to test.

Week 2

Week 1