6. When detailed information and drawings are unavailable, photographs become an important tool in documenting a building. With an interest in photography and image manipulation, I have been researching photogrammetry in an effort to recreate scenes from a combination of sources including movies, home videos, news clips, and stock footage. Using Photosynth and the Photosynth Toolkit, I have extracted images from these sources to create meshes and ultimately 3D models of sites from around the world. These include prominent landmarks such as the Pantheon, the Colosseum and the Statue of Liberty to more organic forms such as Mount Rushmore. With development of this technology, 3D models of entire cities or states could be automatically generated by collecting images from Google, online photo collections, news footage and film. With the development of 3D models, a deeper understanding of the space can be gained.
7.a. Statue of Liberty - 3D Model (3DS Max)
b.Mount Rushmore point cloud
c.St Georges Presbyterian Church, Castlereagh Street, Sydney
EMPIRE STATE BUILDING - http://photosynth.net/view.aspx?cid=2fafb0c6-785f-4699-a111-91f2aa00cb53 / http://photosynth.net/view.aspx?cid=f3788b72-77e3-4ee3-a613-8408e9c3ffd4 http://youtu.be/pVNL3ux1SFI
00 21 13 - 00 21 42 - Arch de Triumph/ Eiffel Tower/ Statue of Liberty 002504 - 002509 - Big Ben 002528-002546 Statue of Liberty 002557-002559 Buckingham Palace 003538-003543 Buckingham Palace 003734-003735 Buckingham Palace
005556-005604 White House 012120-012132 Mount Rushmore 012326-012334 Mount Rushmore
Draft Report Layout -Contents Page -Text Page -2 alternate image styled pages (mid chapter and end chapter)
Scanning through the movies has been very time consuming, with no guarantee of good footage so i began to explore alternatives.
Moving on from the idea of gathering all the footage from film, i began looking for youtube videos (HD) and aerial shots on stock footage sites like https://www.videoblocks.com
this is some footage captured of the Statue of Liberty
Look at: 1. Men in Black Guggenheim (The first scene featuring Will Smith chasing down an alien starts near Grand Central Station, and then he follows the alien up into the white spiral of the Guggenheim Museum. ) 2. Statue of Liberty - Will Smith helps deliver a baby alien squid, is actually Liberty Park in Jersey City, opposite the Statue of Liberty. 3. The Lincoln Memorial and National Mall. (Forrest Gump is another such movie which makes good use of landmarks in the Carolinas, Washington DC and Los Angeles as part of the plot. The Lincoln Memorial and the National Mall are famous attractions in Washington DC which have been featured in many movies, including Mr Smith Goes to Washington, National Treasure and Wedding Crashers.Los Angeles landmarks in Forrest Gump include the Santa Monica Pier, Grauman's Chinese Theater and the historic Ambassador Hotel which was demolished in 2006) 4. Dark Knight (Chicago) 5. Vertigo (San Francisco) 6. When Harry Met Sally (New York) 7. Empire State Building which, according to IMDB, has starred in 77 movies, 4 television shows, 3 video games and 8 short films. Empire State Building played pivotal roles include both the 1933 and latest versions of King Kong, An Affair to remember, and Sleepless in Seattle.
Britain's most famous spy, James Bond, generally spends little time in London, other than to receive his orders from his boss 'M'. However, some of the films do feature locations in the city. These include On Her Majesty's Secret Service (1969) in which George Lazenby as Bond visits the College of Arms and For Your Eyes Only (1981), in which Roger Moore experiences a hair-raising helicopter flight over the Docklands area. In the more recent Pierce Brosnan films, the Secret Service's headquarters are identified as being the new MI6 building on the River Thames at Vauxhall. The 1999 film The World Is Not Enough opens with an extended boat chase from the MI6 building down the river to the Millennium Dome, while in Die Another Day (2002) Bond visits a secret base in a disused Underground station, and makes a rare trip to his club Blades. The 1967 version of Casino Royale makes extensive use of London locations, including 10 Downing Street, Whitehall, Trafalgar Square (with Nelson's Column replaced by a flying saucer) as well as the Changing of the Guard outside Buckingham Palace.
Met with Jeremy on Wednesday. Notes: talk to Josh about reducing the focus to one landmark on film purely because of the time factor spent scrubbing through films at 8X and then only extracting one still image of a landmark, or a blurry image. Use a combination of film and stock to produce synth Develop one landmark to the point of 3D printing (export as .stl)
Looking for Alabrandi Mission Impossible II Our Lips are Sealed Lantana Dirty Deeds Garage Days Finding Nemo Godzilla: Final Wars Strictly Ballroom The Combination
1. Introduction 1.1 Overview 1.2 Aims and Objectives 1.3 Motivation and Purpose
2. The Project 2.1 Who is the Client? 2.2 The brief 2.3 The deliverables
3.0 Precedent Studies 3.1 From 2d Base Map to 3d City Model 3.2 An application of Photogrammetry Measuring Technology 3.3 Digitally Documenting D-Day 3.4 An Update on Automatic 3D Building Reconstruction 3.5 Data Generation for CAAD with Digital Photogrammetry 3.6 City model Enrichment
Compile a collection of movies that feature real urban sites. Focus on Rome, London, New York (and Sydney to a lesser degree if there is good material). Consider putting a call out to facebook friends to help, or searching for "Top 10 films set in London" or similar. 60 dvd quality films categorized.
Composite sites together between clips from the constituent films. You will have a show-reel for each specific location, to be used in presentations and for your reconstruction. Avoid video compression as this will negatively effect your reconstructions. Minimum 12 location show-reels.
(experiment with 20/10/5 frames per second to find optimal. Use Quicktime Pro or similar to export frames out.
reconstruct pointclouds, and attempt meshing using the procedure outlined for PhotoSynth toolkit (or alternative approaches). Minimum 4 meshed reconstructions.
Part 1 London 1.Sliding Doors 2. Snatch 3. Sweeney Todd 4. A Fish Called Wanda 5.The Other Boleyn Girl 6.Elizabeth 7.Casino Royale (1967) and new one? 8.Bridget Jones's Diary 9.Secrets & Lies 10.Gaslight 11.Shakespeare in Love 12.The Italian Job 13.Dial M for Murder 14.Bend It Like Beckham 15.The Ladykillers Love, Actually Notting Hill About A Boy Bridget Jones’s Diary Closer V for Vendetta 28 Days Later
Rome 1. The Talented Mr Ripley, Antony Minghella, 1999 2. Eat, Pray, Love (Eat part in Rome) 3. La Dolce Vita (Federico Fellini - 1960) 4. Three coins in the fountain (1950s) 5. Angels and Demons 6. Roman Holiday (1950S) 7. The Pink Panther 2 8. National Lampoon's European Vacation 9. Ocean's Twelve 10. Gladiator
NewYork 1. Sex and the City 2 (2010) 2. Wall Street: Money Never Sleeps (2010) 3. Date Night (2010) 4. 27 Dresses (2008) 5. P.S. I Love You (2007) 6. The Devil Wears Prada (2006) 7. The Interpreter (2005) 8. Spider-Man 2 (2004) 9. Shaft (2000) 10. 13 (2011)
From my initial idea of using photogrammetry to recreate a scene, like Oxford street over a series of years. I have discovered that i need many more photos than what is available. This led to investigating film, tv and other media as a source for images. This idea could be used to inform design decisions in architecture, recreate scenes from an earlier point in time to be used and manipulated for film, or to document historical changes over time for council records or educational purposes.
At this point i am in the process of reconstructing high definition models a scene from a movie. This process raises a few issues like if you are extracting still images from film, how much of that content is actually real? Are you reconstructing a scene that is already a 3d model? I have reconstructed some test scenes to point clouds, ready to be taken to the next stage as 3d models. From here, these models could be used in applications such as google maps or a series of animations that could show changes over time, possibly combining real video footage and the recreated scene, this way it would be possible to blend past and the future elements.
Crows Nest Shop Reconstruction (point cloud)
Town Hall Reconstruction (Point cloud)
Church Reconstruction (Point cloud)
Finally a breakthrough! Here we have a model in MeshLab.
When it came to the dense reconstruction, with HD images, it stayed at the above point for 24 hours and didnt move. Im assuming it ran out of memory.
Study 6 - ISPRS Journal of Photogrammetry and Remote Sensing - City model Enrichment - Philip D. Smart, Jonathan A. Quinn, Christopher B. Jones
This paper explores how freely available sources of georeferenced Web 2.0 information can be used for automated enrichment of 3D city models. There are many motivations for the production of these models, including:
visualising landscapes (planning and development)
virtual tours and games
radio communications network planning
providing detailed information about an individuals immediate environment through the use of Augmented Reality techniques.
Using these techniques, the information and geometry of nearby features can be overlaid on live video or still images of the environment. It is brought to my attention that the effectiveness of these 3D models depends not so much on visualisation but upon good quality annotation and information that can be associated with the model. There may be a need to link these 3D models to other sources for a richer experience, these may include Wikipedia where there is a large number of georeferenced articles about buildings, places, and other landmarks. OpenStreetMap and Geonames gazetteer are other resources.
http://www.citygml.org
"
The combination of the 2D and 3D models results in a well segmented set of buildings which are accurate to the ground plan, problems matching the two at times is a concern and methods to resolve this are being examined, for example using fuzzy matching techniques that take into account the proximity of multiple point references from the different sources and the identification of potentially important building geometry.
Study 5 - Data Generation for CAAD with Digital Photogrammetry - Horst A. Breyer, Andre Streilein
This is a paper from 1991, however still valid in terms of fundamental ideas. I thought it would be worthwhile having a good look into it to see what the motivations were then and also what was available and where it was heading.
It seems that because of the technology available, there was a lot more thought put into which photographs to take and how many to take. The position of the cameras was all determined manually and lines were computed between points. The prcess showed potential at this point of time, and accuracy could be achieved with large well defined features. Manual and semi automatic measurements could already be achieved. Still it is unlikely today that it can be a fully automated process.
Study 4 - An Update on Automatic 3D Building Reconstruction - Norbert Haala and Martin Kada
Photogrammetry has been around for around 2 decades. This paper looks at current approaches in order to comprehensively elaborate on the reconstruction methods and their principles from polyhedral building objects which represent the blueprint of a building and the roof to more detailed facade geometries from terrestrial data collection.
It seems to be that fully automated image understanding is hard to solve, semi automatic components are usually required to support the recognition of very complex buildings. Suitable image matching software (like what i am using -Photosynth), can generate 3d point clouds and at an accuracy, reliability and detail which was only feasible by LiDAR measurements.
The interactive visualisations of 3d city models were open to the public by applications such as Google Earth and Bing MAps. These visualisations are very coarse and limited to roof structures and planer facades. A considerable number of approaches have been developed to collect 3D building models at acceptable effort for complete city area, but such models feature flat facades and distinctive roof structures. This data is available to a large group of users (via Google Earth) and frequently maintained and used for planning purposes. Facade is however, the most identifying feature of a building usually and the problem is that this is emitted from these models or is very flat. I want to develop this in my project and perhaps think of ways that these facades can be recorded at greater detail.
"Applications like Google Street View or Microsoft Streetside use images collected from such mobile mapping systems to generate panoramic views of urban areas at street level. In this context, these views supplement the oblique images from airborne platforms. Georeferenced video streams as collected from mobile mapping systems can alternatively be used for more advanced real-time visualizations. As an example, large scale 3D reconstructions of street scenes are e.g. required for car navigation systems. Such an approach, which aims at the generation of realistically textured surface models at video frame rates, is e.g. described by Cornelis et al., 2008 " - N. Cornelis, B. Leibe, K. Cornelis and L. Van Gool, 3D urban scene modeling integrating recognition and reconstruction, International Journal of Computer Vision78 (2–3) (2008), pp. 121–141. Full Text via CrossRef | View Record in Scopus | Cited By in Scopus (38)Cornelis et al. (2008).
Study 3 - Digitally Documenting D-Day: The use of close range digital photogrammetry at Pointe Du Hoc - Richard Burt, Ozgur Gonen
Robert Warden, Vinod Srinivasan
this is a good case study of a project using photogrammetry to digitally document the command post at one of the most historical sites of the D-Day landings: Point Du Hoc. A website will be created for the project allowing an interactive tour. The methods used to collect survey data for both the production of 2d Historic American Building Survey drawings and for the 3D digital model are described.
3D laser scanning - ability to create very accurate 3D representations but the amount of raw data recorded can be unmanageable, especially for outdoor scenes.
Image based modelling techniques - inexpensive and can capture objects of any size. Some systems use computer visual techniques such as computational stereopsis to automatically determine the structure of a scene from multiple photos but are limited by the strength of the stereo algorithms used. Multiple tools/software may have to be used to achieve the desired results., this i am noticing more and more. They speak of a program called Facade which is a research prototype.
An important element to consider is lighting. If the photos are taken at different times of the day, different days, different times of the year, the colours and lighting will not match from photo to photo causing inconsistencies in the final output. These can be altered but slightly to match more but will take some time and skill to match the shadows and highlight. The dark areas make it difficult to identify corners etc ie in a dark room.
Obtaining 3D measurements from the images is done by marking the same point on the building on at least 3 images taken from different positions, and then when sufficient information is collected, computer software is able to locate theta 3D position of the cameras. Once the camera positions are located, a series of calculations locate the position of all marked points and create a 3D model. Below is an example of the workflow.
Study 2 - An application of Photogrammetry Measuring Technology to Parametric Modeling of Korean Traditional Wooden Structure - Han-Seon Bae, Ma-rie Kim, Soo-Young Shin, Hyun-A Kim, Chae-Shin Yoon
This paper is about constructing a 3D mole of a Korean traditional wooden structure, with the purpose benign to obtain dimensions. This was done using PhotoModeler, which extracts information from a series of photos. All you need is one measurement to act as a constrain and then the other dimensions can be generated. The possibilities of this are numerous. I can imagine if you were developing a townhouse, you could measure the street length of one and have the program generate the surrounding buildings automatically. It would also be helpful in documentation processes and restoration projects.
Using PhotoModeler, like any photogrammetry software, relies heavily on the quality of photographs (resolution and position of cameras) in order to get good results.
Study 1 - From 2d Base Map to 3d City Model - Matti Arponen - City Survey Division, City of Helsinki, Finland
Laser-scanning and digital photogrammetry have been used in collecting 3d information on map objects in Helsinki. These methods have been applied in mapping out dated map features and in producing 3D models manually, semi-automatically, and automatically. It is noted that it is as much a software development project as it is a 3D city model project.
Some of the problems encountered in this effort (and some i can relate to) include:
finding quality data sufficient for 3D
how should this be connected to the bigger picture of the cities CAD base
Is there enough height information
how can the skills of 2D draftsmen translate to 3D
how is the model to be updated
what level of detail is to be modelled
In this case there were 3 models at 3 scales: a rough city model, a precise city model, and a textured precise city model.
The textured and precise city models have been produced by using a combination of facade photographs and manual and automatic modelling techniques.
The products used in this instance are TerraModeler, TerraScan and TerraPhoto, which re under constant development.
Gathering height information is a big project. Height modelling by an operator using stereo photogrammetric work or interpreting laser scanned points (ground level, buildings, edges of the street elements as break lines would take up to 25 years in this case. Laser scanning as a method to gathering info is being experimented. The problem with these methods are, unlike stereo mapping where only a few points are targeted and measured, the scan picks points which may be of no interest, like humans, traffic lights etc so it may take some time to use this information. The benefit of coarse is the level of detail.
This week i have focussed more on research and the precedent studies....mainly because my computer has been out of action. I have resolved a few issues that i have had over the past couple of weeks but technology has not been kind to me lately. Seeing how i cannot write much on progress.....below are my troubleshooting notes and research i have found helpful. The blog is turning out to be quite a good way of organising my ideas and progress.
notes:
make sure firewall settings on low
UAC off
try on uni lab computers
check the image files are .jpg not .JPG (if you have JGP.key files generated it will not work as it is expecting .jpg.key
"file not found" would seem the process hasn't worked to that point
the openGL error could be to do with the video card
This week i have spent hours and hours experimenting with ways to complete the PhotoSynth process using the PhotoSynth ToolKit 7. There seems to be a glitch with the toolkit that prevents the geometry being formed. Not a lot of progress on paper, but many frustrating hours in front of the computer.
I hope that when i can get the process happening smoothly, the project at the moment has a heavy focus on the process rather than the final product, i'm hoping this will turn around soon and i can start experimenting with some output.
It is interesting that different stages work differently on my 2 computers and some not at all. It has been another tedious week, tricky staying motivated when there are these technical issues which are hard to move past.
This morning I met with Jeremy before the scheduled studio meeting to discuss general progress. It was noted that i will be updating my weekly reports when i get the project on track. It was also recommended that we each need to be spending 12-16 hours a week, i am sitting in the lower part of this range and that will be increased when i finalise the focus of my project.
Some areas i am looking at are:
Hyde Park Barracks
Oxford Street
Reservoir Paddington
I need to do a few tests of an object, a building and maybe a series of buildings.
also do a precedent study....something related to what you are doing. Critique it...what is good about it and what can be improved. What can be learnt, what i endeavour to do with it. It can be bad example and pinpoint how you would do it differently. The precedent studies should take up a third of your final report. Do 1 or more each week
I managed to pull together a few tests with Photosynth this week. I took some photos of a car showroom in artarmon and discovered some initial problems. This gave me a bit of direction and a starting block.
I met with Josh on Thursday for a tutorial on how to take the right images which was beneficial. We started off in town hall and then shortly got the boot so headed to a church on Castlereigh St. I then processed these using Photosynth and the results were pleasing.
After discussing developments with Josh, we have started looking into film as a source of still images to capture and compare areas of interest over the years. This is beneficial as a lot of images can be pulled from film. Yet to test.
