Artikel “Digital Storytelling – a plea for memorable experiences in museums” in der EXPOTIME! 11/12 2018

Case Study: MusOS, a ground-breaking operating system for museums currently been developed to use in the exciting FFA Museum, opening in 2019.

By Wolfgang Bernecker, Sascha Lorenz, Michael Schneitter-Venier

Contact for further information: Wolfgang Bernecker, info@futureact.eu

The Fliegermuseum / Fahrzeugmuseum Altenrhein (FFA) Museum in Switzerland is due to open in July 2019. It will showcase a new IoT (Internet of Things) platform, as well as a custom-built Museum Operating System, called MusOS. MusOS has been designed to provide guests with exceptional and individual museum experiences. Exhibits will interact with visitors directly via augmented reality, sound and light effects, video messages, information screens and puzzles.

The foundation of this new museum technology has been universally developed, however has the potential to feel tailor-made when adapted for use by other museums. The FFA Museum predicts that the continuous development of this platform will see huge improvements in experience, relevance, individualisation or tours curated by the user themselves.

Moreover, the aim was to improve infotainment for guests, increase the quality of research and obtain a higher rate of returning visitors. This novel concept for the museum world is expected to deliver. Furthermore, the system is so flexible it allows customisation, the addition of new stories and changes to the exhibits without having to touch the basic hardware and infrastructure.

Today there is an ongoing concern surrounding the topical of going «digital», especially in conjunction with automatic algorithms of digitisation. Nevertheless, in the context of museum experiences, we must remember that digital technology is only a methodology. The MusOS uses sensors to monitor actuators to provide an enhanced user experience and algorithms for intelligent actions or mobile phones for output and networking.

The use of any of these components is neither good nor bad. However, the aims and objectives behind the choice must decide whether digitising the installation offers a benefit or if it could disrupt from the experience. Thus, there must always be a meaningful idea before embarking on a digital system like MusOS and the FFA’s IoT infrastructure.

Based on a core vision the project team decided to implement the development of three main components: a database optimised for museums, an infrastructure adapted for the building and an app, which is intuitive for visitors.

The aim – to create a storytelling exhibit

The FFA Museum’s main aim was simple. As the only flying and driving museum in central Europe, almost all their exhibits are functional. However, there was an issue – Only a small number of their visitors could afford to enjoy a flight or a ride.  To meet these guest’s expectations, they required another attraction to substitute for this interactivity and experience.

Additionally, exhibits are moved or updated daily in the hangar, which meant info boards, screens and other displays proved to be inappropriate on the floor space.

The desired solution was a system, which conveys the correct information, relevant to the exhibit to the visitor via a smart device. To reduce the investment cost of the hardware, guests would be encouraged to use their own devices to communicate with and control the flow of stories in the museum. Or put simply, the exhibits tell the story directly to visitors as soon as they encounter each other.

Key questions for consideration

Once the core idea was defined, it opened a series of questions:

• Does an exhibit tell the same story to every visitor? Not really.
• Are there other ways to communicate the story? We could use video, text, fragrance, light and augmented reality.
• Will the story be experienced individually or as a group? We don’t want to create another solution which is focused on the individual like the classical audio guide. We should create a shared experience for families, couples and pupils.
• Is the FFA Museum the only museum, which could benefit from a storytelling exhibit? Definitely not.

The digital storytelling approach

Have you ever wished that as you approach an exciting exhibit its stories come to life with ease, imagination and creative interactivity? Now, imagine yourself entering our museum, you are presented with a choice of virtual companion: Jochen Rindt the former Formula 1 star, Coco Chanel the celebrated fashion designer, Richard Feynman a renowned scientist famous for his presentations or Andrei Tuploew the well-known aircraft engineer. Depending on your chosen guide the exhibits tell a different aspect of its story.

For example, at the Rolls-Royce Avon engine exhibit, Jochen Rindt introduces you to the sound of the time which you experience through a loudspeaker. Andrej Tupolew will show you the assembly of the engine in augmented reality. With Coco Chanel, you will witness the fashion of the former pilots which is visible on the screen. Richard Feynman will explain to you the limits of jet engines. Thus, the presentation of exhibit is different and unique, depending on which guide you have selected.

Via the storytelling approach relevant access to each exhibit increases, participants can choose what is relevant for them. For bystanders the story is also perceptible or can be shared. In the case of a family visiting the museum, it is easy to imagine a father, after studying the jet engine in AR, explaining the details to his daughter using the AR on his mobile phone. Or the son may start the sound of the engine to show his mum and repeat for a second or third time… Welcome to the world of contextual computing.

Contextual computing

In 2013 Forbes Magazine first described Contextual Computing as the sixth, seventh and eighth human senses. Therefore the industry is not brand new. Innovative first movers made initial experiences, leaving the package of technology and knowledge ripe for further growth. Unfortunately, the number of experts in the field is not yet large enough to conquer all industries at once.

The museum with its variety of stories and content offers a perfect fit for demonstrating the power of the technology. Following Forbes’ statement, the sixth sense represents the opportunity to see beyond the exhibit itself. Today it is often not enough to present a dusty old exhibit. The raw materials of the 21st century are stories.

The intuitive connection to relevant information is the seventh sense. From the pool of stories linked to an exhibit, it is important to choose which content can create the most personalised experience of the exhibit. This creates a magical bond that captivates and fascinates a visitor. The eighth sense describes the interaction and use of a variety of different technologies.

Recent developments in communication standards and the introduction of IPv6 became the milestones the industry needed to develop further. The standardisation of interfaces and the growing trend of using the Internet as a central protocol for machine-to-machine communication builds company’s trust in R&D investments and allows the creation of devices which are autonomous and interchangeable. Therefore, owners are not tied to an individual manufacturer and can easily integrate and upgrade to the latest technology if desired, without changing their infrastructure.

From know-how to performance

Detailed benchmarks and a feasibility study were the starting points for the FFA Museum. Wolfgang Bernecker and Bernhard Vonier, one of the museum’s owners, went back to the vision statement using inspiration and information from these studies and concluded that the integration of an IoT platform in the museum was the direction to pursue.

Chatbots were the chosen format for machine-to-human or exhibit-to-visitor interface, as this seemed the most appropriate based on the popularity of accepted tools like WhatsApp. The interaction of everyone and everything around the museum was then formalised in the Visitor Journey and refined in following steps.

The Visitor Journey describes the core activities visitors can experience during a museum tour around different zones. This also includes preparation for the visit with the help of the website and a review of the visit using the museum’s app. However, the focus is on the exhibits themselves. What stories are available and worth including and who are they relevant to?

It was important that the core idea behind each zone in the journey was free of technical jargon and only based on the aim, the objective and the benefit.

Ideally the Visitor Journey is generated by the museum or with museum consultants or agencies.

Then it’s time to link the stories to all the available devices in the space. Here the focus is on MusOS. The Museum Operating System connects the benefits, story data and devices throughout the building. The software architects then integrated interfaces with a focus on IT standards to create a sustainable structure. Today, MusOS is compatible with Raspberry PIs, KNX, DALI, Android and IOS operating systems. Other standards such as Dante or DMX can be quickly implemented.

As a result, the scenographer can concentrate on associating the content with the exhibits and doesn’t need to be concerned with the technical details. Loudspeakers, screens, lights and action points are virtually linked to the corresponding exhibit via the position of the exhibit in the museum. As soon as the story is made available for guests, the system will automatically connect to the necessary output devices.

With the Visitor Journey in hand and a full understanding of the technical interfaces of MusOS available, building related engineers can now easily translate the functional description into the technical components and parameters which need to be installed in the desired space. This applies to the electrical, lighting, media and telecommunications technology. The process is fairly simple.

Crucial advice for all museums who embark on a similar project is to remain focused on the functional description. This will always provide a common level of understanding between the museum and the professional engineer, creating an opportunity where everybody can input their own specialities.

Many see the implementation of technology as the most important task in modernising a museum. Interestingly, this misjudgement often leads to failure of installations. The technology only provides the approach and the tools but is never the full solution.

Now MusOS needs to be fed. Here the FFA Museum differentiates between scenographers and data managers. These are often two separate groups of people, each with their own strengths. The data manager takes care of hierarchy, structure and completeness. Storytelling, design, empathy – the user experience is the concern of the scenographer.

Many museums will work with volunteers and enthusiasts, which is great. In scenography, working with external students for smaller stations can be refreshing. Larger stations are a business field for agencies and professionals. Particularly in the field of scene design obtaining an external perspective is a great way to reduce the risk of organisational and operational blindness.

The building of MusOS

MusOS’s development was initiated by the FFA Museum and was divided into three projects. Central is the CMS (Content Management System). The essential technical aspect is the format of objects, object types and hierarchies. Additional fields for an object type or even a completely new type can be implemented in fractions of a second without any programming knowledge or necessity of a software company. Object types in the museum are of course all exhibits, but we extended the concept.

Also, a supplier or an event are potential object types. Thus, the database is truly multifunctional. In addition to the core function of storytelling, there are now additional areas to provide behind the scenes support. For example, the administration of an event is also included allowing managers to keep an overview of invoices linked to their specific events, guides can get free access to the museum at relevant times, social media and welcome screens can be automatically published, or areas in the museum can be temporarily closed.

Another feature is the easy linking of related data with tags. Thus, exhibits can be grouped in addition to the classical hierarchy, if for example thematic affinities exist. This offers an invaluable advantage when displaying data records on the Internet or during a museum tour.

Just imagine all the museums relevant visitor data arranged on a sphere instead of a table. Thus, the user has more opportunities to explore information just by turning the virtual sphere in any direction.

The second ingredient of MusOS is the IoT area of the software. The links to all electronic devices in the museum is managed here. As a result, the stations and zones come to life and the building becomes an experience space instead of a lifeless shell.

Today, algorithms can be intelligent enough, so visitors do not notice if a failure of a single component occurs. For example, if a screen loses connectivity, default content is displayed, or a backup action is initiated.

The final component of MusOS is the app or the batch. A batch is an electronic pendant for children or people who are less interested or don’t have the capabilities to actively interact. It allows these guests to follow a pre-defined tour.

When using the app, the exhibits will, pro-actively contact visitors, ideally on their own smartphone, as soon as they get closer. At present this seems to be the most natural form of communication between man and machine.

Each exhibit will now choose the story it believes has the highest relevance for the individual visitor, based on their choice of virtual guide and their previous actions.

In parallel a random generator and a check of previous visits influences the decision. As a result, the app will select from an initial pool of 9 hours of stories to create a unique tour lasting approx. 90 minutes. This algorithm also allows returning visitors to have new experiences each time.

Tablets and batches are available for visitors to use if preferred. A dedicated batch for children has been designed to enable exhibits to respond even without user input. It is also possible to give children a higher priority in specific zones.

To perform, each exhibit has multiple chatbots. As soon as a visitor is offered a story via chat – for example «Do you want to hear the sound of my engine?” – other potential stories will be blocked or delayed. This is necessary because simultaneous engine sounds or noise of several exhibits side by side would impact other experiences. However, the non-availability of actions is only visible for MusOS and administrators but cannot be seen by guests.

The three software packages and its flexibility make MusOS a system that is far beyond existing and standard software. It also offers efficiency for the administration and marketing of the museum. The storytelling opportunities, simplicity and the ability to deliver tailored content for each individual museum are core to MusOS.

Further development

Where will MusOS go next? Over the next few months MusOS will be developed into an off-the-shelf product, available for use by other museums. Simultaneously, work will be done to refine the content-related usage and the intelligence of the algorithms.

The University of Applied Sciences Vorarlberg are working with the FFA Museum to prepare Bachelor and Masters courses, such as «Digital Storytelling», «Augmented Reality Processes» and «a User-Curated Museum» which will be designed to deliver new input and expertise based on a scientific foundation.

Michael Schneitter-Venier, focusing on museum pedagogy is an experienced teacher and student of Intermedia. His goal is to increase infotainment and the sustainability of knowledge transfer in museums.

The focus for all parties involved in this project is to understand the system’s intuitiveness to minimise potential inhibitions both quickly and effectively.

Michael says, “Once we are armed with the research, we expect to achieve an optimal balance between system control and freedom of decision-making. We will never have a visitor who will consume the entire offer of the museum during one visit – this consists of about 9 hours of prepared content. By offering a good selection of virtual guides, using a random generator and a series of further criteria, the team wants to optimise algorithms quickly – based on real customer data.

It is predicted that should a visitor feel very limited in their choice they will quickly become frustrated. Through close observation and analysis of their behaviour, we can implement quick and vital adaptations.

In time we will learn from continued usage and alongside the growth of the contextual computing industry, we will gain new insights which will provide invaluable input into future updates of the system.”

One area for further research is the fascinating topic of a «user-curated museum». Instead of tours based upon virtual personality profiles, i.e. a racing driver or a pilot, what about making visitors creators of their own tours. The challenge would be to provide the user with enough assistance and support to enable them to make meaningful selections. A guidance system and recommendations of suitable content could easily be offered and improved in the near future.

Undoubtedly, it would be a bonus to gift friends with a unique entrance ticket adding an individually composed tour – or think about a teacher who prepares a visit for his class. A tool for user-curated tours would offer new opportunities for educationists. 

Gamification

Gamification is a new buzzword in the IoT industry. If a museum visit were to offer a platform for a digitally interested audience to immerse themselves in a game, this would be a game changer for visitor interaction. For the development process the FFA Museum takes inspiration from the 1990s classic computer game «The Secret of Monkey Island», the Escape Room trend and games à la Mario Party by Nintendo. With the increasing spread of networked rooms full of 8-sense-experiences, we can expect further improvements incorporating gained knowledge over the coming years.

Summary and Outlook

Digital systems make sense in museums if the rationale behind them is based on a clear aim and provides an answer to specific tasks. The FFA Museum sees the exhibit «only» as a foundation of the experience. By linking the general data with the stories surrounding the exhibit, the museums events, the position of the guests in the building and the building infrastructure, MusOS creates a new layer of potential experiences and thus a new level of quality.

Based on this foundation, internal and external scenographers can stage the exhibits and present content according to the user’s preferences.

In addition, MusOS improves the efficiency of the everyday running of the museum by managing events, contacts and linking to the museum’s website and eShop. The administrative processes are actively reduced, and each record entered can be accessed for multiple uses.

Once the basic data is input, the steps to tailor the system are readily available. Furthermore, MusOS is programmed for returning visitors, who can expect to experience new content whenever they visit. Finally, human guides – still seen as the number one choice for storytelling – can supplement their performance using the IoT devices to demonstrate their explanations.

The system is also built in logical bundles with defined interfaces. If for example, our beloved smartphones are replaced by another technology, then the entire concept does not have to change. An upgrade to the new technology device is enough, all other soft-and hardware remains functional. This considerably increases the longevity of investments.

A world class development will be to win the visitor as a «gamer». Allowing the user to immerse themselves into an augmented world that equally demands, entertains and educates. This will be an extraordinary experience based on passion, fun and knowledge. This new format is expected to meet the wishes of the younger generation and creates new attractiveness. Gamification is already being researched at the FFA Museum in cooperation with colleges and universities. The FFA Museum and Contexagon – the software architects – are breaking new ground here as well.

At the start of the project, the museum decided it would be fruitless to tailor the operating system to the FFA-Museum requirements alone. A transferable concept coordinating the functional scope of other museums made much more sense, not only for cost reasons. Processes were created to allow knowledge and experience sharing as an operator and to support other museums in their own implementation. Through other museums use further development and optimisation of the software is ensured. Even cross-museum projects and decentralised museum tours or interaction possibilities are manageable and are currently being researched. Naturally, the FFA Museum expects to see its own advantages deriving from these developments in the near future.

Wolfgang Bernecker led the FFA Museum as Museum Manager until October 2018 and was responsible for the Vision, Basic Concept and Visitor Journey. Today, he continues to support the technological and conceptional setup of the museum. As a self-professed pioneering spirit, he now pursues a series of projects in the fields of digital, marketing and lighting.

Sascha Lorenz is co-founder of Contexagon. The company passionately and expertly develops efficient software solutions around the topics Context Awareness Computing, Augmented Reality, Virtual Reality, Machine Learning, Mobile Apps and Big Data. Contexagon were awarded the contract for the development of MusOS for the FFA Museum.

Michael Schneitter-Venier teaches English, Physics, Chemistry, Learning Support and Integration, Information and Communication Education. As a master student in the department «Intermedia» at the University of Applied Sciences Vorarlberg he focusses on museum education. At the FFA Museum, he applies and refines his research.