TF-NOC flash presentation NIIF/Hungarnet NOC János Mohácsi director of network and multimedia (mohacsi@niif.hu) net-admin@niif.hu 28 May 2013
Outline NIIF Network NIIF/Hungarnet organisation NOC structure and services NOC tools and integration Frontend Communication tools Documentation Conclusion
NIIF Networks member institutions ~5670 member institutions locations ~ 1.5 million users ~3000 member institutions Universities, academic/research institutions Primary and secondary schools Public collections: libraries and museums Some hospitals Access speed from 512Kbps to 100 Gbps 601 monitored core devices 10799 monitored access devices (data from 5/2013) Hungarnet bottom up association
HBONE+ backbone + Access Infrastructure DF (IRU) Leased lambdas Licensed ptp microwaves ADSL DOCSIS Cable MetroEthernet Leased lines Leased Services from 8 telcos 4.
HBONE+ Network elements Optical backbone lots of effort 69 locations, ~3200km Shared usage of fibre with HEC Local loops Countrywide DWDM system - Alcatel-Lucent PSS-1830 Optical services from user to user Most advanced 80 channel system with ROADMs efficient operation 10G, 40G and 100G services Operated by NIIF NOC Countrywide IP/MPLS system Cisco ASR9000 MEF Carrier Ethernet/MPLS SP Platform 92 (184) Gbps slot capacity Green system Operated by NIIF NOC IP devices Mostly Cisco platform 5.
Hybrid Network Services Lightpath Service Dedicated Network capacity for projects several demands Point-to-point circuits λod L2 VPN, managed Ethernet - Finegrained capacity provisioning with routers Traditional IP: IPv4/6, unicast/ multicast, L3 VPN, with QoS and SLA 6.
NIIF Organisation and NOC 8 departments (~60) 4 technical + financial + project + partner relations + international relations ~26 services NOC is part of Department of Network Infrastructure 15 full-time member ~ 15 supporting member at regional centers (remote-hand) 1-2 hours support/months
NOC organisation and services Tier0: Call center 24x7 basic question dispatching Tier1: Working Days 8-18h 2T1 in the morning & 2 T1 in the afternoon Tier2: 24x7 weekly shift on duty engineer 8-18 in case of emergency, 18-8+weekend - major maintenance, major failure NOC staff (15+1)~ 5 T1, 2 T2, 5 T1&T2, 3 field, 1 boss Services: Helpdesk for institutions (Network, Multimedia services, application services) Connectivity management (NIIF Network, GEANT, BIX, private peerings) Monitoring and network maintenance Network configuration (QoS, filtering, routing, IP allocation) T2 members also planning and development
NOC tools Strategy Everything should use eduid (Shibboleth) Use/develop opensource components Tools: net-admin@niif.hu - heavily relying on e-mail communication + phone Drupal based ticketing with federated AAI- http://drupal.org/project/ticketing Integrated Information system IIR own development DWDM PhM Weathermap modified http://www.network-weathermap.com/ Cricket modified http://cricket.sourceforge.net/ - also for optical monitoring Nagios with plenty of own developed plugins modified http://www.nagios.org -migrating to Icinga Syslog-ng + Netcool for log analysis Rancid http://www.shrubbery.net/rancid/ + routercvs with cvsweb Nfdump and NfSen - http://sourceforge.net/projects/nfsen/ Small scripts (perl, python, bash, php ): Integration, Reporting & Automation FreeRadius and Cisco AR (phasing out) Performance: NDT network diagnostics, iperf OOB OTDR, power meter, cleaning tools, Ethernet traffic generators Integration: Web portal for NOC and for users with CRM portal
Ticketing system Self made: drupal, shibboleth authen?ca?on, role based authorisa?on Communica?on: web and mail Used for tracking troubles, tasks, leased line crea?on/dismantling
Front end What types of users are using your network and services? end-users individual users NIIF member institutions system administrators and senior representatives Outside partners Other departments How does the SLAs or agreements you have with your customers look like? SLA in the agreements 99.5%, but achieved 99.97% Membership type organisation discussion/collective work Which tools are used to communicate with and keep track of users?
Frontend /2
Frontend /2
Frontend /2
Communication tools To communicate with and keep track of users E-mail, phone, videoconference, monthly meeting Trouble ticketing system user can submit and keep track of tickets IIR Other groups e-mail, phone Issue tracking systems (redmine) IIR Trouble ticketing system Instant messaging Outside NIIF e-mail, phone IIR Trouble ticketing system + external TT
IIR informations Data about the end-point Location Responsible persons Devices - contracts Integrated with existing systems - Cricket, nagios, ticketing 16. oldal
IIR informations Data about the end-point Location Responsible persons Devices - contracts Integrated with existing systems - Cricket, nagios, ticketing 17. oldal
IIR informations Data about the end-point Location Responsible persons Devices - contracts Integrated with existing systems - Cricket, nagios, ticketing 18. oldal
Documentation Documented information mostly hungarian Service descriptions Procedures Advices, hints Modifications Config procedures Contacts Tools Moinmoin wiki IIR Word document Do you have any Best Practice Documents available to share??
THANKS?