OSC Overview
Clusters
Creating an application to be run on OSC systems will require an understanding of the underlying system architecture. Currently, AweSim web applications are hosted on Web nodes residing on the Glenn cluster, and tasks are primarily solved by submitting to nodes on the Oakley cluster. Please familiarize yourself with OSC's supercomputers.
Login Nodes
Each supercomputer has a small number of nodes set aside as login nodes. These nodes are shared among users and are used for non-computationally intensive tasks like system text editing and file management. Much of the modern web development process takes place in a command line environment, and at OSC, that environment is the login node. For additional details, see Login Environment at OSC
File Systems
There are several methods of file storage available to app developers. The primary place where you will be storing and developing applications will be your home directory on the NFS file system. This file system is shared, which means that it is available to you across all clusters. There is a storage limit of 500GB available to you in this space and it is considered permanent storage, which means that you can safely store your project files without risk of deletion. For projects larger than 500GB and 1 million files, users can make a request for a Project Directory, which is capable of handling more and larger files, but for a fixed term of time. In addition to the shared file systems, each compute node has local disk space available for temporary file storage while a job is running. This is accessed from within a running job via the $TMPDIR command. Local storage is the fastest way to access files during a compute job, and it is recommended that compute jobs with a large amount of disk I/O requests copy files into the node's local storage when a job begins, and back to the home directory when the computation is finished. Finally, OSC provides a high-performance Lustre file system for large read and write operations. This system can be accessed manually at /fs/lustre/<YourUserName> or from within a job by using the $PFSDIR variable. For more information, see Available File Systems
The Batch System
High Performance Computing is performed by requesting resources through the batch system. This involves submitting a script to our scheduler via a qsub command. The job is then queued until the requested resources become available. Once it begins, the job then runs until it has completed or the wall time is reached. AweSim apps generally follow this workflow by providing a user-friendly frontend to this process. Job variables can be obtained from a user via a familiar web form and used to complete a job script template. Most of the command line functions you will need to create a Rails application are provided to you via the pbs-ruby and osc-machete Ruby gems. It is strongly recommended that you understand the underlying process for batch submission and job management. If you are new to the batch system, please review the tutorials provided by OSC at Batch Processing at OSC.
Additional Resources
The Ohio Supercomputer Center offers occasional training workshops and seminars. Please consult the OSC calendar to learn about upcoming training events.
Development: Getting Started
OSC provides Ruby gems and example applications to application developers via a private Github organization. Contact OSC Help to gain access to these repositories. In addition to this guide, the official documentation resource for developers can be found at apps.awesim.org/devapps under the 'Documentation' tab.
Your Apps
Apps that you create will appear at the top of the developer dashboard at apps.awesim.org/devapps. Any apps that you have created will be displayed at the top of this page. During development, your apps should be located under the awesim_dev directory of your home folder and individual apps should be developed in folders named rails1 - rails9 to function properly from the dashboard.
Two ways to build an app
1. Create a new app using a generator
2. Modify an existing application
In both cases, you will want to do your development on our Glenn cluster, which provides the same environment as our web hosting nodes. To get started, open a terminal and issue the following commands to log in to the cluster, navigate to the development path, and load the proper software packages:
$ ssh glenn.osc.edu
$ cd awesim_dev
$ module load rubyApps that you create in the $HOME/awesim_dev/ location are only visible to you on the dashboard. This is the location for developing and testing your Rails/Rack applications to be used in the AweSim environment.
1. Building an app using a generator
An AweSim-specific app template is available to create a blank Rails application containing AweSim Gems and styles. You can create this application using the Rails generator and adding the path to the template.
$ rails new demo_app -m /nfs/01/wiag/PZS0645/awesim/rails-application-template/awesim.rbOnce the application has been generated, you will need to change the directory name to one of rails1 - rails9 so that it will properly load from the dashboard.
$ mv demo_app rails1The app is now accessible from the dashboard.
Please see the new app tutorial for guidance on building a new application.