Applying a Real-time CORBA ORB for Avionics Mission Computing schmidt@cs.wustl.edu Washington University, St. Louis www.cs.wustl.edu/çschmidt/tao4.ps.gz Sponsors Boeing and CDI/GDIS
Mission Computing Design Requirements and Forces Integrate real-time scheduling/dispatching in ORB and I/O subsystem for Boeing military aircraft product families i.e., Harrier (AV/8b), F-15, and F/A-18 Provide all applications with real-time capabilities Both method-oriented and event-oriented applications Meet deterministic and statistical QoS requirements i.e., minimize latency, context switching, priority inversion, and non-determinism Washington University, St. Louis 1
Motivation for CORBA for Mission Computing www.cs.wustl.edu/çschmidt/corba.html Benefits Simplify distribution by automating Object location and activation Parameter marshaling Demultiplexing Error handling Provide foundation for higher-level services Washington University, St. Louis 2
The ACE ORB (TAO) www.cs.wustl.edu/çschmidt/tao.html TAO Overview A real-time, high-performance ORB Leverages ACE Runs on POSIX, Win32, RTOSs Related work U. RI, Mitre QuO at BBN ARMADA at U. Mich. Washington University, St. Louis 3
The ADAPTIVE Communication Environment (ACE) www.cs.wustl.edu/çschmidt/ace.html ACE Overview Concurrent OO networking framework Ported to C++ and Java Runs on RTOSs, POSIX, and Win32 Related work x-kernel SysV STREAMS Washington University, St. Louis 4
ACE Statistics ACE contain é 135, lines of C++ Over 15 person-years of effort Ported to UNIX, Win32, MVS, and embedded platforms e.g., VxWorks, LynxOS, psos Large user community www.cs.wustl.edu/çschmidt/aceusers.html Currently used by dozens of companies Bellcore, Boeing, Ericsson, Kodak, Lockheed, Lucent, Motorola, SAIC, Siemens, StorTek, etc. Supported commercially www.riverace.com Washington University, St. Louis 5
Applying TAO to Avionics Mission Computing www.cs.wustl.edu/çschmidt/oopsla.ps.gz Domain Challenges Periodic deterministic (and some statistical) real-time deadlines COTS infrastructure Open systems Related work Deng, Liu, and J. Sun 96 Gopalakrishnan and Parulkar 96 Wolfe et al. 96 Washington University, St. Louis 6
TAO s Real-time ORB Endsystem Architecture Solution Approach Integrate RT dispatcher into ORB endsystem Support multiple request scheduling strategies e.g., RMS,EDF,andMUF Requests ordered across thread priorities by OS dispatcher Requests ordered within priorities based on data dependencies and importance Washington University, St. Louis 7
Server Request Reception Use-case Synopsis I/O subsystem uses port numbers to demux requests to queues and RT threads per rate group A Reactor demuxes/dispatches requests for each rate group Washington University, St. Louis 8
Event Channel Reception Use-case Synopsis Event Channel threads handle event importance and dependencies I/O subsystem and ORB Core handle priorities Washington University, St. Louis 9
ORB Latency and Priority Inversion Experiments C C 1 Requests Client 11 1...... C n Servants Object Adapter ORB Core I/O SUBSYSTEM Server ATM Switch 1 11 1 Ultra 2 Ultra 2 www.cs.wustl.edu/çschmidt/rt-perf.ps.gz SCHEDULER RUNTIME Vary ORBs, hold OS constant Methodology 1 high-priority client 1..n low-priority clients Server uses thread-per-priority Highest real-time priority for high-priority client Lowest real-time priority for low-priority clients Washington University, St. Louis 1
ORB Latency and Priority Inversion Results Latency per Two-way Request in Milliseconds 52 48 44 4 36 32 28 24 2 16 12 8 4 CORBAplus High Priority CORBAplus Low Priority MT-ORBIX High Priority MT-ORBIX Low Priority minicool High Priority minicool Low Priority TAO High Priority TAO Low Priority 1 5 1 15 2 25 3 35 4 45 5 Number of Low Priority Clients Synopsis of results TAO s latency is lowest TAO avoids priority inversion i.e., high-priority client always has lowest latency Overhead stems from concurrency and connection architecture e.g., synchronization and context switching Washington University, St. Louis 11
ORBJitterResults Definition Variance from average latency 3 Jitter in milliseconds 25 2 15 1 5 1 5 1 15 2 25 3 Number of Low Priority Clients 35 4 45 5 CORBAplus Low Priority CORBAplus High Priority MT-ORBIX Low Priority MT-ORBIX High Priority minicool Low Priority minicool High Priority TAO Low Priority TAO High Priority Synopsis of results TAO s jitter is lowest and most consistent CORBAplus jitter is highest and most variable Washington University, St. Louis 12
User-level and Kernel-level Locking Overhead 7 16 User Level Lock Operations per Request 6 5 4 3 2 1 1 client server 94 231 199 216 175 599 Kernel Level Lock Operations per Request 14 12 1 8 6 4 2 client server 13 8 4 4 14 41 TAO minicool CORBAplus MT ORBIX ORBs Tested TAO minicool CORBAplus MT ORBIX ORBs Tested TAO is carefully designed to minimize memory allocation and locking Washington University, St. Louis 13
Real-time OS/ORB Performance Experiments [P ] C C 1 Requests [P] Priority Client [P ] 1...... C n [P ] n 11 1 Servants Object Adapter [P ] [P ] 1... ORB Core S S 1 S n I/O SUBSYSTEM Server Pentium Pro www.cs.wustl.edu/çschmidt/rt-os.ps.gz... [P ] n SCHEDULER RUNTIME Vary OS, hold ORBs constant Methodology 1 high-priority client 1..n low-priority clients Server uses thread-per-priority Highest real-time priority for high-priority client Lowest real-time priority for low-priority clients Washington University, St. Louis 14
Real-time OS/ORB Performance Results Latency per Two-way Request in Milliseconds 1 9 8 7 6 5 4 3 2 1 Solaris x86 High Priority LynxOS High Priority VxWorks High Priority Windows NT High Priority Solaris x86 Low Priority LynxOS Low Priority VxWorks Low Priority Windows NT Low Priority Synopsis of results RTOS s provide lowest latency RTOS s minimize priority inversion ORB (TAO) provides low latency and avoids priority inversion i.e., highpriority client always has lowest latency 1 5 1 15 2 25 3 35 4 45 5 Number of Low Priority Clients Washington University, St. Louis 15
Real-time OS/ORB Jitter Results Jitter in milliseconds 1.4 1.2 1.8.6.4.2 1 5 1 15 2 25 3 Number of Low Priority Clients 35 4 45 5 VxWorks Low Priority VxWorks High Priority LynxOS Low Priority LynxOS High Priority Solaris x86 Low Priority Solaris x86 High Priority Windows NT Low Priority Windows NT High Priority Definition Standard deviation from average latency Synopsis of results Some RTOS s provide low jitter ORB (TAO) doesn t introduce jitter Washington University, St. Louis 16
Real-time OS/ORB CPU Utilization Experiments C Requests Client 11 1 Intel x86 Utilization Servants Object Adapter ORB Core I/O SUBSYSTEM Server Ethernet Switch SCHEDULER RUNTIME 11 1 SPARC www.cs.wustl.edu/çschmidt/rt-os.ps.gz Vary ORBs, hold OS constant Methodology 1 client thread 2 server threads 1 thread services client 1 thread factors prime numbers Washington University, St. Louis 17
Real-time OS/ORB CPU Utilization Results Percentage of Utilization of CPU 9 8 7 6 5 4 3 2 46 CPU Utilization IDLE Time 8 8 54 2 2 52 48 Synopsis of results RTOS s provide highest effective utilization ORB (TAO) processing uses ç2% of the CPU 1 Solaris x86 LynxOS VxWorks Windows NT RTOSs Tested Washington University, St. Louis 18
Concluding Remarks TAO is currently used at Boeing for avionics mission computing Initial flight dates are mid-summer 1998 Extensive benchmarks demonsrate it is possible to meet stringent performance goals with real-time CORBA e.g., for Boeing, target latency for CORBA oneway operations is 15 çsecs for 1 Mhz PowerPC running over MVME 177 boards Technology transfer to commercial vendors via OMG RT SIG and DARPA Quorom program Washington University, St. Louis 19