Saturday, February 11, 2012

FreeBSD

Credits:

http://www.freebsd.org
All content here is for information purpose only. The whole content and many more information could be found at FreeBSD web site

What is FreeBSD?

FreeBSD is an advanced operating system for x86 compatible (including Pentium® and Athlon™), amd64 compatible (including Opteron™, Athlon™64, and EM64T), ARM, IA-64, PowerPC, PC-98 and UltraSPARC® architectures. It is derived from BSD, the version of UNIX® developed at the University of California, Berkeley. It is developed and maintained by a large team of individuals. Additional platforms are in various stages of development.

History

FreeBSD development began in 1993 with a quickly growing, unofficial patchkit maintained by users of the 386BSD operating system. This patchkit forked from 386BSD and grew into an operating system taken from U.C. Berkeley's 4.3BSD-Lite (Net/2) tape with many 386BSD components and code from the Free Software Foundation. After two public beta releases via FTP (1.0-GAMMA on September 2, 1993, and 1.0-EPSILON on October 3, 1993), the first official release was FreeBSD 1.0, available via FTP on November 1, 1993 and on CDROM on December 30, 1993. This official release was coordinated by Jordan Hubbard, Nate Williams and Rodney W. Grimes with the name thought up by David Greenman. Walnut Creek CDROM agreed to distribute FreeBSD on CD and gave the project a machine to work on along with a fast Internet connection, which Hubbard later said helped stir FreeBSD's rapid growth. A "highly successful" FreeBSD 1.1 release followed in May 1994.

However, there were legal concerns about the BSD Net/2 release source code used in 386BSD. After a lawsuit between UNIX copyright owner at the time Unix System Laboratories and the University of California, Berkeley, the FreeBSD project re-engineered most of the system using the 4.4BSD-Lite release from Berkeley, which, owing to this lawsuit, had none of the AT&T source code earlier BSD versions had depended upon, making it an unbootable operating system. Following much work, the unencumbered outcome was released as FreeBSD 2.0 in January 1995.

FreeBSD 2.0 featured a revamp of the original Carnegie Mellon University Mach virtual memory system, which was optimized for performance under high loads. This release also introduced the FreeBSD Ports system, which made downloading, building and installing third party software very easy. By 1996 FreeBSD had become popular among commercial and ISP users, powering extremely successful sites like Walnut Creek CD-ROM (a huge repository of software that broke several throughput records on the Internet), Yahoo! and Hotmail. The last release along the 2-STABLE branch was 2.2.8 in November 1998.[9] FreeBSD 3.0 brought many more changes, including the switch to the ELF binary format. Support for SMP systems and the 64-bit Alpha platform were also added. The 3-STABLE branch ended with 3.5.1 in June 2000.

Linux compatibility

Most software that runs on Linux can run on FreeBSD without the need for any compatibility layer. FreeBSD nonetheless still provides a compatibility layer for several other Unix-like operating systems, including Linux. Hence, most Linux binaries can be run on FreeBSD, including some proprietary applications distributed only in binary form. Examples of applications that can use the Linux compatibility layer are StarOffice, the Linux version of Firefox, Adobe Acrobat, RealPlayer, Oracle, Mathematica, Maple, MATLAB, WordPerfect, Skype, Wolfenstein: Enemy Territory, Doom 3 and Quake 4 (though some of these applications also have a native version). No noticeable performance penalty over native FreeBSD programs has been noted when running Linux binaries, and, in some cases, these may even perform more smoothly than on Linux. However, the layer is not altogether seamless, and some Linux binaries are unusable or only partially usable on FreeBSD. This is often because the compatibility layer only supports system calls available in the historical Linux kernel 2.4.2. There is support for Linux 2.6.16 system calls, available since FreeBSD 7.0 and enabled by default since FreeBSD 8.0. However, there is currently no support for running 64-bit Linux binaries.

Cutting edge features

FreeBSD offers advanced networking, performance, security and compatibility features today which are still missing in other operating systems, even some of the best commercial ones.

Powerful Internet solutions

FreeBSD makes an ideal Internet or Intranet server. It provides robust network services under the heaviest loads and uses memory efficiently to maintain good response times for thousands of simultaneous user processes.

Advanced Embedded Platform

FreeBSD brings advanced network operating system features to appliance and embedded platforms, from higher-end Intel-based appliances to Arm, PowerPC, and shortly MIPS hardware platforms. From mail and web appliances to routers, time servers, and wireless access points, vendors around the world rely on FreeBSD's integrated build and cross-build environments and advanced features as the foundation for their embedded products. And the Berkeley open source license lets them decide how many of their local changes they want to contribute back.

Run a huge number of applications

With over 20,000 ported libraries and applications, FreeBSD supports applications for desktop, server, appliance, and embedded environments.

Easy to install

FreeBSD can be installed from a variety of media including CD-ROM, DVD, or directly over the network using FTP or NFS. All you need are these directions.

FreeBSD is free

The BSD Daemon

While you might expect an operating system with these features to sell for a high price, FreeBSD is available free of charge and comes with full source code. If you would like to purchase or download a copy to try out, more information is available.

Contributing to FreeBSD

It is easy to contribute to FreeBSD. All you need to do is find a part of FreeBSD which you think could be improved and make those changes (carefully and cleanly) and submit that back to the Project by means of send-pr or a committer, if you know one. This could be anything from documentation to artwork to source code. See the Contributing to FreeBSD article for more information.

Even if you are not a programmer, there are other ways to contribute to FreeBSD. The FreeBSD Foundation is a non-profit organization for which direct contributions are fully tax deductible. Please contact board@FreeBSDFoundation.org for more information or write to: The FreeBSD Foundation, P.O. Box 20247, Boulder, CO 80308, USA.

FreeBSD offers many advanced features.

No matter what the application, you want your system's resources performing at their full potential. FreeBSD's focus on performance, networking, and storage combine with easy system administration and excellent documentation to allow you to do just that.

A complete operating system based on 4.4BSD.

FreeBSD's distinguished roots derive from the BSD software releases from the Computer Systems Research Group at the University of California, Berkeley. Over ten years of work have been put into enhancing BSD, adding industry-leading SMP, multithreading, and network performance, as well as new management tools, file systems, and security features. As a result, FreeBSD may be found across the Internet, in the operating system of core router products, running root name servers, hosting major web sites, and as the foundation for widely used desktop operating systems. This is only possible because of the diverse and world-wide membership of the volunteer FreeBSD Project.

FreeBSD provides advanced operating system features, making it ideal across a range of systems, from embedded environments to high-end multiprocessor servers.

FreeBSD 7.0, released February 2008, brings many new features and performance enhancements. With a special focus on storage and multiprocessing performance, FreeBSD 7.0 shipped with support for Sun's ZFS file system and highly scalable multiprocessing performance. Benchmarks have shown that FreeBSD provides twice the MySQL and PostgreSQL performance as current Linux systems on 8-core servers.

  • SMPng: After seven years of development on advanced SMP support, FreeBSD 7.0 realizes the goals of a fine-grained kernel allowing linear scalability to over 8 CPU cores for many workloads. FreeBSD 7.0 sees an almost complete elimination of the Giant Lock, removing it from the CAM storage layer and NFS client, and moving towards more fine-grained locking in the network subsystem. Significant work has also been performed to optimize kernel scheduling and locking primitives, and the optional ULE scheduler allows thread CPU affinity and per-CPU run queues to reduce overhead and increase cache-friendliness. The libthr threading package, providing 1:1 threading, is now the default. Benchmarks reveal a dramatic performance advantage over other UNIX® operating systems on identical multicore hardware, and reflect a long investment in SMP technology for the FreeBSD kernel.
  • ZFS filesystem: Sun's ZFS is a state-of-the-art file system offering simple administration, transactional semantics, end-to-end data integrity, and immense scalability. From self-healing to built-in compression, RAID, snapshots, and volume management, ZFS will allow FreeBSD system administrators to easily manage large storage arrays.
  • 10Gbps network optimization: With optimized device drivers from all major 10gbps network vendors, FreeBSD 7.0 has seen extensive optimization of the network stack for high performance workloads, including auto-scaling socket buffers, TCP Segment Offload (TSO), Large Receive Offload (LRO), direct network stack dispatch, and load balancing of TCP/IP workloads over multiple CPUs on supporting 10gbps cards or when multiple network interfaces are in use simultaneously. Full vendor support is available from Chelsio, Intel, Myricom, and Neterion.
  • SCTP: FreeBSD 7.0 is the reference implementation for the new IETF Stream Control Transmission Protocol (SCTP) protocol, intended to support VoIP, telecommunications, and other applications with strong reliability and variable quality transmission through features such as multi-path delivery, fail-over, and multi-streaming.
  • Wireless: FreeBSD 7.0 ships with significantly enhanced wireless support, including high-power Atheros-based cards, new drivers for Ralink, Intel, and ZyDAS cards, WPA, background scanning and roaming, and 802.11n.
  • New hardware architectures: FreeBSD 7.0 includes significantly improved support for the embedded ARM architecture, as well as preliminary support for the Sun Ultrasparc T1 platform.

FreeBSD has a long history of advanced operating system feature development; you can read about some of these features below:

  • A merged virtual memory and filesystem buffer cache continuously tunes the amount of memory used for programs and the disk cache. As a result, programs receive both excellent memory management and high performance disk access, and the system administrator is freed from the task of tuning cache sizes.
  • Compatibility modules enable programs for other operating systems to run on FreeBSD, including programs for Linux, SCO UNIX, and System V Release 4.
  • Soft Updates allows improved filesystem performance without sacrificing safety and reliability. It analyzes meta-data filesystem operations to avoid having to perform all of those operations synchronously. Instead, it maintains internal state about pending meta-data operations and uses this information to cache meta-data, rewrite meta-data operations to combine subsequent operations on the same files, and reorder meta-data operations so that they may be processed more efficiently. Features such as background filesystem checking and file system snapshots are built on the consistency and performance foundations of soft updates.
  • File system snapshots, permitting administrators to take atomic file system snapshots for backup purposes using the free space in the file system, as well as facilitating background fsck, which allows the system to reach multiuser mode without waiting on file system cleanup operations following power outages.
  • Support for IP Security (IPsec) allows improved security in networks, and support for the next-generation Internet Protocol, IPv6. The FreeBSD IPsec implementation includes support for a broad range of accelerated crypto hardware.
  • Out of the box support for IPv6 via the KAME IPv6 stack allows FreeBSD to be seamlessly integrated into next generation networking environments. FreeBSD even ships with many applications extended to support IPv6!
  • Multi-threaded SMP architecture capable of executing the kernel in parallel on multiple processors, and with kernel preemption, allowing high priority kernel tasks to preempt other kernel activity, reducing latency. This includes a multi-threaded network stack and a multi-threaded virtual memory subsystem. Beginning with FreeBSD 6.x, support for a fully parallel VFS allows the UFS file system to run on multiple processors simultaneously, permitting load sharing of CPU-intensive I/O optimization.
  • M:N application threading via pthreads permitting threads to execute on multiple CPUs in a scalable manner, mapping many user threads onto a small number of Kernel Schedulable Entities. By adopting the Scheduler Activation model, the threading approach can be adapted to the specific requirements of a broad range of applications.
  • Netgraph pluggable network stack allows developers to dynamically and easily extend the network stack through clean layered network abstractions. Netgraph nodes can implement a broad range of new network services, including encapsulation, tunneling, encryption, and performance adaptation. As a result, rapid prototyping and production deployment of enhanced network services can be performed far more easily and with fewer bugs.
  • TrustedBSD MAC Framework extensible kernel security, which allows developers to customize the operating system security model for specific environments, from creating hardening policies to deploying mandatory labeled confidentiality of integrity policies. Sample security policies include Multi-Level Security (MLS), and Biba Integrity Protection. Third party modules include SEBSD, a FLASK-based implementation of Type Enforcement.
  • TrustedBSD Audit is a security event logging service, providing fine-grained, secure, reliable logging of system events via the audit service. Administrators can configure the nature and granularity of logging by user, tracking file accesses, commands executed, network activity, system logins, and a range of other system behavior. Audit pipes allow IDS tools to attach to the kernel audit service and subscribe to events they require for security monitoring. FreeBSD supports the industry-standard BSM audit trail file format and API, allowing existing BSM tools to run with little or no modification. This file format is used on Solaris and Mac OS X, allowing instant interoperability and unified analysis.
  • GEOM pluggable storage layer, which permits new storage services to be quickly developed and cleanly integrated into the FreeBSD storage subsystem. GEOM provides a consistent and coherent model for discovering and layering storage services, making it possible to layer services such as RAID and volume management easily.
  • FreeBSD's GEOM-Based Disk Encryption (GBDE), provides strong cryptographic protection using the GEOM Framework, and can protect file systems, swap devices, and other use of storage media.
  • Kernel Queues allow programs to respond more efficiently to a variety of asynchronous events including file and socket IO, improving application and system performance.
  • Accept Filters allow connection-intensive applications, such as web servers, to cleanly push part of their functionality into the operating system kernel, improving performance.

FreeBSD provides many security features to protect networks and servers.

The FreeBSD developers are as concerned about security as they are about performance and stability. FreeBSD includes kernel support for stateful IP firewalling, as well as other services, such as IP proxy gateways, access control lists, mandatory access control, jail-based virtual hosting, and cryptographically protected storage. These features can be used to support highly secure hosting of mutually untrusting customers or consumers, the strong partitioning of network segments, and the construction of secure pipelines for information scrubbing and information flow control.

FreeBSD also includes support for encryption software, secure shells, Kerberos authentication, "virtual servers" created using jails, chroot-ing services to restrict application access to the file system, Secure RPC facilities, and access lists for services that support TCP wrappers.

Applications for FreeBSD

Experience the possibilities with FreeBSD

FreeBSD can handle nearly any task you would expect of a UNIX® workstation, as well as many you might not expect:

FreeBSD is a true open system with full source code.

There is no doubt that so-called open systems are the requirement for today's computing applications. But no commercial vendor-supplied solution is more open than one which includes full source code to the entire operating system, including the kernel and all of the system daemons, programs, and utilities. You can modify any part of FreeBSD to suit your personal, organizational, or corporate needs.

With its generous licensing policy, you can use FreeBSD as the basis for any number of free or commercial applications.

FreeBSD runs thousands of applications.

Because FreeBSD is based on 4.4BSD, an industry-standard version of UNIX, it is easy to compile and run programs. FreeBSD also includes an extensive packages collection and ports collection that bring precompiled and easy-to-build software right to your desktop or enterprise server. There is also a growing number of commercial applications written for FreeBSD.

Here are some examples of the environments in which FreeBSD is used:

  • Internet services. Many Internet Service Providers (ISPs) find FreeBSD ideal, running WWW, Usenet news, FTP, Email, and other services. Ready-to-run software like the Apache web server or the ProFTPD FTP server make it easy to set up a business or community-centered ISP. Of course, with FreeBSD's unbeatable networking, your users will enjoy high speed, reliable services.
  • X Window workstation. From an inexpensive X terminal to an advanced X display, FreeBSD works quite well. Free X software (X.Org™) comes with the system. nVidia offers native drivers for their high-performance graphics hardware, and the industry standard Motif® and OpenGL® libraries are supported. Both the KDE and GNOME desktop environments enjoy full support and provide office suite functionality, with further good functionality available in the OpenOffice.Org and TextMaker products.
  • Networking. From packet filtering to routing to name service, FreeBSD can turn any PC into a Internet firewall, email host, print server, PC/NFS server, and more.
  • Software development. A suite of development tools comes with FreeBSD, including the GNU C/C++ compiler and debugger. Java® and Tcl/Tk development are also possible for example, and more esoteric programming languages like Icon work just fine, too. And FreeBSD's shared libraries have always been easy to make and use. You can also choose from a wide range of popular and powerful editors, such as XEmacs and Vim.
  • Net surfing. A real UNIX workstation makes a great Internet surfboard. FreeBSD versions of Firefox and Opera are available for serious web users. Surf the web, publish your own web pages, read Usenet news, and send and receive email with a FreeBSD system on your desktop.
  • Education and research. FreeBSD makes an excellent research platform because it includes complete source code. Students and researchers of operating systems or other computer science fields can benefit greatly from such an open and well-documented system.
  • And much more. Accounting, action games, MIS databases, scientific visualization, video conferencing, Internet relay chat (IRC), home automation, multiuser dungeons, bulletin board systems, image scanning, and more are all real uses for FreeBSD today.

FreeBSD is an operating system that will grow with your needs.

Though FreeBSD is free software, it is also user supported software. Any questions you have can be posted to hundreds of FreeBSD developers and users simply by e-mailing the freebsd-questions@FreeBSD.org mailing list.

FreeBSD also has a worldwide group of programmers and writers who fix bugs, add new features and document the system. Support for new devices or special features is an almost constant development process, and the team keeps a special eye out for problems which affect system stability. FreeBSD users are quite proud of not only how fast but how reliable their systems are.

What experts have to say . . .

``FreeBSD handles [our] heavy load quite well and it is nothing short of amazing. Salutations to the FreeBSD team.''

-- Mark Hittinger, administrator of WinNet Communications, Inc.


Internetworking

FreeBSD was designed for the Internet

FreeBSD includes what many consider the reference implementation for TCP/IP software, the 4.4 BSD TCP/IP protocol stack, thereby making it ideal for network applications and the Internet. FreeBSD 7.0 also includes the reference implementation of SCTP, thus making it an ideal platform for telephony and other data streaming applications.

FreeBSD supports all standard TCP/IP protocols.

Like most UNIX® systems, the FreeBSD operating system enables you to

  • Serve static and dynamic web content over HTTP
  • Share filesystems with NFS
  • Distribute network information with NIS
  • Handle and provide email services over SMTP, IMAP and POP3
  • Support remote logins over SSH and rsh
  • Do remote SNMP configuration and management
  • Serve files with FTP
  • Resolve Internet hostnames with DNS/BIND
  • Route packets between multiple interfaces, including PPP and SLIP lines
  • Use IP Multicast services (the MBONE)
  • Provide services over IPv6

FreeBSD lets you to turn a PC into a World Wide Web server, mail server or an Usenet news relay with included software. Using the included SAMBA software you can even share filesystems or printers with your Microsoft® Windows® machines and, with the supplied PCNFS authentication daemon, you can support machines running PC/NFS. FreeBSD also supports Appletalk and Novell client/server networking (using an optional commercial software package), making it a true "Intranet" networking solution.

FreeBSD also handles TCP extensions like the RFC-1323 high performance extension, plus SLIP and dial-on-demand PPP. It is an operating system suitable for a home-based net surfer as well as a corporate systems administrator.

FreeBSD's networking is stable and fast.

If you need an Internet server platform that is reliable and offers the best performance under heavy load, then consider FreeBSD. Here are just a few of the companies that make use of FreeBSD every day:

  • Walnut Creek CDROM ran one of the most popular FTP servers on the Internet, ftp.cdrom.com, exclusively on FreeBSD for many years. It was a single FreeBSD machine supporting 6000 connections, and capable of transferring more than 30 terabytes (as of June, 1999; yes that is terabytes!) worth of files every month to more than 10 million people.
  • Yahoo Inc. runs the ultimate index of the Internet, serving scads of daily net surfers with information about the World Wide Web. Yahoo, as well as the companies that advertise on Yahoo, rely on FreeBSD to run reliable and responsive web servers.
  • Netcraft is the leading researcher of web server software usage on the Internet. They use FreeBSD and Apache to power their website, and FreeBSD/Perl for all their Internet data collection.

FreeBSD makes an ideal platform for these and other Internet services:

  • Company-wide or world-wide WWW service
  • Proxy WWW service
  • Anonymous FTP service
  • Enterprise file, print and mail services
  • Routers, firewalls and intrusion detection systems

The FreeBSD ports collection contains ready-to-run software that makes it easy to set up your own Internet server.

High performance and security.

The FreeBSD developers are as concerned about security as they are about performance. FreeBSD includes kernel support for IP firewalling, as well other services, such as IP proxy gateways. If you put your corporate servers on the Internet, any computer running FreeBSD can act as a network firewall to protect them from outside attack.

Encryption software, secure shells, Kerberos, end-to-end encryption and secure RPC facilities are also available.

Furthermore, the FreeBSD team is proactive in detecting and disseminating security information and bug reports with a security officer and ties to the Computer Emergency Response Team (CERT).

What experts have to say...

``FreeBSD ... provides what is probably the most robust and capable TCP/IP stack in existence ...''


---Michael O'Brien, SunExpert August 1996 Volume 7 number 8.


Advocacy Project

Much of the success which surrounds FreeBSD is due to people advocating its use to their friends, colleagues, and employers.

This page provides links to more information to help you do this.

Mailing lists

Web resources

  • *BSD Myths

    Describes and debunks some of the myths that surround the *BSD projects.

  • On Advocating FreeBSD and the Halloween memo

    Jordan Hubbard's response to the infamous Halloween memo, posted to the FreeBSD-Advocacy mailing list (and republished with his permission).

  • FreeBSD in the Press

    Contains many links to articles that have appeared which mention FreeBSD.

  • FreeBSD CD Artwork

    Artwork for a FreeBSD 4.6 CD cover by Oliver Rapp, available for non-commercial use.

Sites using FreeBSD

  • Hosting Providers Performance by Netcraft is tracking the reliability of major webhosting services, many of them are using FreeBSD.
  • The Open Directory Project's goal is to produce the most comprehensive directory of the web by relying on a vast army of volunteer editors.
  • BSDCan, the annual BSD Conference held in Ottawa, Canada.
  • EuroBSDCon, the annual BSD Conference in Europe.

Marketing Materials

This page contains presentations, white papers, and other marketing materials for FreeBSD.

White Papers

Thinking of using FreeBSD in a project? Finding it hard to convince your boss, the CTO, the CEO? Read through these real life examples of FreeBSD successes with shipping products, then give them to the decision makers at your company.

Presentations

Flyers

  • What is FreeBSD? (PDF | PostScript)
  • BSD Success Stories (27 pages) (PDF), O'Reilly.

Press Highlights



FreeBSD Art

This page contains miscellaneous FreeBSD art. Suggestions for additions can be sent to www@FreeBSD.org. Please note the usage policy for these graphics.

BSD Daemon

BSD Daemon
Created by Poul-Henning Kamp
Source: /usr/share/examples/BSD_daemon/ on FreeBSD systems.
BSD Daemon  wielding a hammer BSD Daemon waiting  tables
BSD Daemon editing the  news BSD Daemon reading  documentation BSD Daemon  delivering the latest release

Powered by FreeBSD Logos

Powered by FreeBSD Logo Powered by FreeBSD Logo
Powered by FreeBSD Logo Powered by FreeBSD Logo FreeBSD Hardware Partner Logo

FreeBSD The Power To Serve Logo

FreeBSD The Power To Serve Logo

FreeBSD The Power To Serve Logo

FreeBSD The Power To Serve Logo

Old Advertisement Banners

Adv Banner1 Adv Banner2

Graphics Use

The Powered by FreeBSD logos above may be downloaded and displayed on personal or commercial home pages served by FreeBSD machines. Use of this logo or the likeliness of the BSD Daemons for profitable gain requires the consent of Brian Tao (creator of the power logo) and Marshall Kirk McKusick (copyright holder for the BSD Daemon image).

Poul-Henning Kamp's rendering of the BSD Daemon is released under THE BEER-WARE LICENSE. See the README for more information.

Trademarks

The FreeBSD Foundation holds several FreeBSD related trademarks (among them the trademark for the term FreeBSD itself). For more information about these trademarks read the FreeBSD Trademark Usage Terms and Conditions.

FreeBSD Logo

Usage Guideline

FreeBSD is a registered trademark of The FreeBSD Foundation. The FreeBSD logo and The Power to Serve are trademarks of The FreeBSD Foundation.

All images listed under the heading "Resource" are available for use under license from The FreeBSD Foundation.

For more information on how to obtain permission to use the logo, please refer to the FreeBSD Logo Usage Guidelines at The FreeBSD Foundation.

Resource

Standard Logo (fullcolor)

Standard Logo (fullcolor for dark background)

Standard Logo (black and white)

Vector formats

Format: Adobe(r) Illustrator(r), SVG

Sample

NOTE: "freeBSD" text in these images were created based on draft version of logo contest. Correct version of this text should be rendered by one black color, not two colors and first "f" character should be spelled capitalized as "F".

CD/DVD package

Postcard

Wallpaper

Here are sample wallpapers.

Monday, February 6, 2012

HP TippingPoint Next Generation Intrusion Prevention System

What Is It?

HP TippingPoint’s NGIPS uses adaptive intelligence to protect your network from the most advanced and sophisticated attacks, delivering these threat protection capabilities:

Application Awareness and Enforcement

Through HP TippingPoint AppDV, GeoLocation, and RepDV services, we identify and classify applications in real-time and then take action to block attacks and/or families of attacks by geographic origin or destination, at the application layer, independent of port or protocol.

Context Awareness

Provides increased confidence and visibility into alert severity by evaluating user information, GeoLocation, Reputation Awareness, and ThreatLinQ intelligence. Then, by integrating that information into a single console, the HP TippingPoint Security Management System (SMS), you get more actionable event data and better policy decision making.

Content Awareness

We prevent the spread of malware by inspecting inbound and outbound communications for content and executables, giving you the ability to identify and stop malicious traffic that may be communicating with command-and-control servers or attempting to steal user information.

Agile Engine

We easily integrate new services from HP TippingPoint’s DVLabs such as AppDV, WebAppDV and RepDV to ensure you’re protected against new types of threats. New defense techniques are dynamically incorporated into the NGIPS, providing you with investment protection and reduced capital expenditures.

First Generation IPS Features

HP TippingPoint has a long history of providing in-line, real time vulnerability threat protection featuring high inspected throughput and low latency.

What Makes Us Better?

In addition to our award winning NGIPS device, HP TippingPoint provides industry leading intelligence research through our unique HP TippingPoint DVLabs organization. Our DVLabs team defines, develops and delivers our NGIPS security services, including Reputation DV, Application DV, Web Application DV, Digital Vaccine Toolkit, and our Digital Vaccine filter service. When combined with our HP TippingPoint Security Management System (SMS) and ThreatLinQ security portal, you get instant visibility into the global threat environment, and integrated event data, resulting in better policy decision making, better device management, and a better security posture for your network.

Model Description
S660N 750Mbps 5 Gig-T/5 1Gb Fiber Segments IPS (JC019A)
S1400N 1.5Gbps 5 Gig-T/5 1Gb Fiber Segments IPS (JC020A)
S2500N 3Gbps 5 Gig-T/1 10GbE/ 5 1GbE Fiber Segments IPS (JC021A)
S5100N 5Gbps 5 Gig-T/1 10GbE/ 5 1GbE Fiber Segments IPS (JC022A)
S6100N 8Gbps 5 Gig-T/1 10GbE/ 5 1GbE Fiber Segments IPS (JC577A)
S5100N 10Gbps IPS Bundle
S6100N 16Gbps IPS Bundle

Services and Support

HP TippingPoint Networking Services add value to your business The warranties on HP TippingPoint Networking products provide a robust foundation for ongoing network support. HP offers an affordable, comprehensive portfolio of networking services for organizations of all size and is the only vendor in the IT industry that can provide integrated and consistent support for your entire technology infrastructure worldwide.

Linux File System Fsck Testing

Credits:
Jeffrey Layton

FSCK Testing Plan

It has been a while since we started the fsck project to test fsck (file system check) times on Linux file systems. The lengthy delay in obtaining the results is due to the lack of hardware for testing. The original vendor could not spare the hardware for testing. A number of other vendors were contacted and due to various reasons none of them could provide the needed hardware for many, many months if at all. In the end, Henry used his diplomatic skills to save the day, persuading Data Direct Networks to help us out. Paul Carl and Randy Kreiser from DDN contacted me and agreed to provide remote access to the hardware (thank you, DDN!). Paul used a DDN SFA10K-X with 590 disks that are 450GB, 15,000 rpm SAS disks. He used a 128KB chunk size in the creation. From these disks he created a number of RAID-6 pools using an 8+2 configuration (8 data disks and 2 parity disks). Each pool is a LUN that is 3.6TB in size before formatting or actually 3,347,054,592 bytes as reported by "cat /proc/partitions". The LUNS were presented to the server as disk devices such as /dev/sdb1, /dev/sdc1, /dev/sdd1, ..., /dev/sdx1 for a total of 23 LUNs of 3.6TBs each. This is a total of 82.8 TBs (raw). The LUNs were combined using mdadm and RAID-0 to create a RAID-60 configuration using the following command:

mdadm -- create /dev/md1 -- chunk=1024 -- level=0 -- raid-devices=23 /dev/sdb1
/dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1
/dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 /dev/sdn1 /dev/sdo1 /dev/sdp1 /dev/sdq1
/dev/sdr1 /dev/sds1 /dev/sdt1 /dev/sdu1 /dev/sdv1 /dev/sdw1 /dev/sdx1
The result was a file system with about 72TB using "df -h" or 76,982,232,064 bytes from "cat /proc/partitions". A second set of tests were run on storage that used only 12 of the 23 LUNs. The mdadm command is

mdadm -- create /dev/md1 -- chunk=1024 -- level=0 -- raid-devices=12 /dev/sd
The resulting file system for this configuration is about 38 TBs using "df -h".

The server used in the study is a dual-socket, Intel Xeon system with Nehalem processors (E5520) running at 2.27 GHz and an 8MB cache. The server has a total of 24GB of memory, and it was connected to the storage via a Qlogic Fibre Channel FC8 card connected to an FC switch that was connected to the storage. The server ran CentOS 5.7 (2.6.18-274 kernel). The stock configuration was used throughput the testing except for one component. The e2fsprogs package was upgraded to version 1.42, enabling ext4 file systems larger than 16TB to be created. This allows the fcsk performance of xfs and ext4 to be contrasted.

Building the file systems was done close to the default behavior that many system admins will adopt -- using the defaults. The commands for building the file systems are:

  • XFS: /sbin/mkfs.xfs -f /dev/md1
  • EXT4: /sbin/mke2fs -t ext4 -F /dev/md1
Mounting the file systems involved a little more tuning. In the case of XFS, I used the tuning options as stated by Dell, XFS -- rw,noatime,attr2,nobarrier,inode64,noquota. In the case of ext4, the mounting options used are.
defaults,data=writeback,noatime,barrier=0,journal_checksum.

The journal checksum was turned on within ext4 since I like this added behavior.

Step 1: Filling the File System

One of the keys to the testing is how the file system is filled. This can be a very time consuming process because you must create all of the files in some sort of order or fashion. For this testing, fs_mark was used. Ric Wheeler at Red Hat has been using it for testing file systems at very large scales (over 1 billion files). Fs_mark wasn't used for testing the file system in this article, but rather, it is used to fill the file system in a specific fashion. It uses one or more base directories and then creates a specified number of subdirectories underneath them that are filled with files. You might think of this as a single-level of subdirectories. It is much more complicated to create specific subdirectory depths and number of files since that configuration depends on the specific users and situation. You could also use some sort of random approach with the hope that a random distribution approximates a real-world situation. It is virtually impossible to have a representative file system tree that fits most general situations, and the single-level deep directory tree used here should represent one extreme of file systems -- a single subdirectory level.

One of the nice features of fs_mark is that it is threaded so that each thread produces its own unique directory structure with a single layer of subdirectories underneath a base directory that contains a fixed number of files. Fs_mark also allows you to specify the number of files per thread so that you can control the total number of files. Although the server has eight total cores, running eight threads (one per core) it resulted in the OS swapping. When the number of threads is reduced to three, the server did not swap, and the file creation rate was much faster than running eight threads with swapping.

Using three threads causes some issues because it is an odd number. This made it impossible to determine an integer number of files per thread, as using the old file counts was not possible. The number of files per thread was changed to a reasonable integer number that is close to the original numbers of 100,000,000, 50,000,000, and 10,000,000. The numbers chosen were: 105,000,000, 51,000,000, and 10,200,000.

The goal for all fs_mark commands was to fill the file system to the specified number of files while filling about 50 percent of the file system. The following fs_mark command lines were used to fill the file system for 72TB:

  • ./fs_mark -s 400000 -L 1 -S 0 -n 35000000 -D 35000 -N 1000 -t 3 -k -d /mnt/test
  • ./fs_mark -s 800000 -L 1 -S 0 -n 17000000 -D 17000 -N 1000 -t 3 -k -d /mnt/test
  • ./fs_mark -s 4000000 -L 1 -S 0 -n 3400000 -D 3400 -N 1000 -t 3 -k -d /mnt/test

The commands for filling the 38TB file systems were:

  • ./fs_mark -s 200000 -L 1 -S 0 -n 35000000 -D 35000 -N 1000 -t 3 -k -d /mnt/test
  • ./fs_mark -s 400000 -L 1 -S 0 -n 17000000 -D 17000 -N 1000 -t 3 -k -d /mnt/test
  • ./fs_mark -s 2000000 -L 1 -S 0 -n 3400000 -D 3400 -N 1000 -t 3 -k -d /mnt/test

Notice that the number of files per directory is a constant (-N 1000 or 1,000 files).

After the file system was filled using fs_mark, it was unmounted, and the file system check was run on the device. In the case of xfs, the command is,

/sbin/xfs_repair -v /dev/md1

For ext4, the file system check was,

/sbin/e2fsck -pfFt /dev/md1

Notice that the device /dev/md1 was the target in both cases.

Step 2: Running More Fsck Tests

DDN was kind of enough to offer additional testing time so I decided to try some tests that stretched the boundaries a bit. The first test was to create an XFS file system with 415,000,000 files and filling about 40 percent of the file system on the 72TB file system. The second test was to try to increase the fragmentation of the file system by randomly adding and deleting directories using fs_mark for the 105,000,000 file case also on the 72TB file system.

For the first test where 415,000,000 files were created, the original goal was to test 520,000,000 files in five stages of 105,000,000 files (creating 520,000,000 all at once caused the server to swap badly). However, due to time constraints, only four of the five stages could be run (fs_mark ran increasingly slower the more files were on the system). The final number of files created was 420,035,002 which also includes all "." and ".." files on the directories.

For the second test, approximately 105,000,000 files were created on an XFS file system in several steps. A total of five stages were used where 21,000,000 files were added at each stage using fs_mark (a total of 105,000,000 files). In between the stages, a number of directories were randomly removed, and the same number of directories anf files were replaced using fs_mark on randomly selected directories. The basic process is listed below:

  1. Use fs_mark to create 21,000,000 files using,
    • 3 threads of 7,000,000 files each
    • 7,000 directories
    • 1,000 files per directory
  2. Randomly remove 700 directories and their files ("rm -rf")
  3. use fs_mark to add 700 directories with 1,000 files each to 700 randomly chosen existing directories (one directory is added to one existing directory)
  4. Use fs_mark to create 21,000,000 more files (42,000,000 total at this point)
  5. Randomly remove 1,400 directories and their files
  6. Use fs_mark to add 1,400 directories with 1,000 files each to 1,400 randomly chosen existing directories
  7. Use fs_mark to create 21,000,000 more files (63,000,000 total at this point)
  8. Randomly remove 2,100 directories and their files
  9. Use fs_mark to add 2,100 directories with 1,000 files each to 2,100 randomly chosen existing directories
  10. Use fs_mark to add 21,000,000 more files (84,000,000 total at this point)
  11. Randomly remove 2,8000 directories and their files
  12. Use fs_mark to add 2,8000 directories with 1,0000 files each to 2,800 randomly chosen existing directories
  13. Use fs_mark to add the final 21,000,000 files (105,000,000 total at this point)
  14. Randomly remove 3,500 directories and their files
  15. Use fs_mark to add 3,500 directories with 1,000 files each to 3,500 randomly chose existing directories

Because of the random nature of selecting the directories, it is possible to get some directories with many more files than others. However, the total number of files won't be 105,000,000 since of the random nature of selection for deletion and insertion. If we count all fo the files including the "." and ".." files we find that the process created 115,516,127 files.

FSCK Results

The table below lists the file system repair times in seconds for the standard matrix cases as specified in the previous article but with the new number of files. These times include all steps in the file system checking process.

File System Size (in TB) Number of Files (in Millions) XFS - xfs_repair time (Seconds) ext4 - fsck time (Seconds)
72
105
1,629
3,193
72
51
534
1,811
72
10.2
161
972
38
105
710
3,372
38
51
266
1,358
38
10.2
131
470

The FSCK time for the additional tests are listed below:

  • 415,000,000 file case:11,324 seconds
  • Fragmented case: 676 seconds

Notice that the 415,000,000 case took 6.95 times longer than the 105,000,000 file case even though it had four times as many files. During the file system check the server did not swap, and no additional use of virtual memory was observed.

The "fragmented" case is interesting because it took less time to perform the file system check than the one-level directory case. The original case took 1,629 seconds and the fragmented case took only 676 seconds -- about 2.5 times faster. Time did not allow investigating why this happened.

In the next article in this seriesHenry writing about his observations of the results. Please be sure to post your comments about these testing results.

A Big Thank You

At first glance it seemed simple a vendor could provide about 80TB to 100TB of raw storage connected to a server for testing, but this turned out not to be the case. It was far more difficult than anticipated. I would be remiss if I didn't thank the people who made this possible: Of course Henry Newman for pushing various vendors to help if they could. Thanks go to Paul Carl and Randy Kreiser from DDN who greatly helped in giving me access to the hardware and helped with the initial hurdles that crop up. Thanks also to Ric Wheeler who answered several emails about using fs_mark and about Linux file systems in general. He has been a big supporter of this testing from the beginning. Thanks also to Andres Dilger from Whamcloud who provided great feedback and offers of help all of the time.

Jeff Layton is the Enterprise Technologist for HPC at Dell, Inc., and a regular writer of all things HPC and storage.

Henry Newman is CEO and CTO of Instrumental Inc. and has worked in HPC and large storage environments for 29 years. The outspoken Mr. Newman initially went to school to become a diplomat, but was firmly told during his first year that he might be better suited for a career that didn't require diplomatic skills. Diplomacy's loss was HPC's gain.

Reference:

http://www.enterprisestorageforum.com