Link zur deutschen Version: 3D-Apps mit AR-Funktionalitäten für die Lehre im Stahlbetonbau
Introduction
Learning the fundamentals of structural engineering involves not only the understanding of mathematical formulae and mechanical models but also their interaction with the real-world 3D space. Our chair had a good experience with the introduction of web applications to help our students understand the main principles taught in our structural concrete classes. These applications, though, mainly focus on the abstract representation of the content through 2D drawings, diagrams and text. Additionally, they offer the possibility to interact with the underlying mechanical models by altering the geometric, material or other properties using sliders as well as textual input fields. While this approach is adequate for many different teaching contents, as shown by 17 applications already in use for our bachelor`s classes, we also found examples that would benefit from a different presentation style. Three drawbacks were identified that hinder the current approach from being adapted for some examples. These are:
- Representation of concepts in an abstract manner using 2D drawings or diagrams. This can complicate the students’ ability to relate the concepts shown to real-life examples.
- Many different input options like sliders, text fields and multiple output figures can be overwhelming for the students.
- The apps are written within different programming environments, which can be a hurdle for inexperienced teaching assistants.
To address these three points, the new generation of apps is based on the following three ideas:
- The representation of geometry in 3D space both in a virtual environment and also anchored in the real world using augmented reality (AR) functionality.
- A clear user/student journey through the application to avoid overwhelming the students by the complexity of the model.
- A simple implementation concept in the backend that enables the easy manipulation of existing examples and the quick creation of new examples.
It has to be emphasized that the newly developed framework for 3D–AR–Apps is not meant to replace the current collection of apps used at our chair. Instead, the goal is to apply these in cases where the current generation of apps is not suitable.
Prototype
Our first example is based on one of the exercises in the module “Structural Concrete I”, dealing with the design of a torsion beam using strut and tie models. The User can access the app with most mainstream browsers on either a mobile device or a computer. To get started, simply visit the provided link or scan the QR code with the camera.
Once the app is running, you can explore its features through the user interface.
For the AR function, grant camera access when prompted, then scan an Aruco 6×6 marker to utilize augmented reality features.
Backend Implementation Framework
The main idea of the framework is to be easily useable and adaptable for app creators who are not familiar with Unity. Therefore, widely used file types were chosen as the input. In the current state of the project implementation, these include the following raw input files:
- 3D-geometry (.3dm)
- Layer information (.xlsx)
- Story information (.xlsx)
- Step descriptions (.markdown)
The framework consists of three main parts. The first step involves converting all raw input files into formats that are easily communicated via the internet, such as OBJ, JSON and CSV. The second step entails designing a Unity project to receive these materials and deserialise them into 3D geometry, diagrams or math equations. A customised navigation system for mouse and touch input in a browser environment has been developed, enabling users to experience the content in 3D space. Additionally, an AR implementation is introduced to help students understand the geometry in the physical space. The third step involves compiling and deploying this app as a web application, which is then hosted on a web server for users to access.If you want to know more about the process of creating a new app, feel free to have a look at the readMe file.
Conclusion
The current version of the app represents the culmination of the team’s ideas and efforts over the past few months. Your participation in testing and providing feedback is crucial for optimisation and for the next phase of the project. We aim to test the stability and user-friendliness of the interactive features, as well as ensure compatibility with a variety of devices before releasing a robust version to students. The main goal of the next phase is to scale up the project. Theoretically, new applications can be created by simply replacing the remote materials.
We invite you to test the first prototype of the app on our website and would be very happy if you would share your thoughts using this feedback questionnaire.
Paul Merz and Wenqian Yang