<commit_before>//! # Custom `derive` pretending to be functional procedural macros on Rust 1.15
//!
//! This crate enables creating function-like macros (invoked as `foo!(...)`)
//! with a procedural component,
//! based on both custom `derive` (a.k.a. *Macros 1.1*) and `macro_rules!`.
//!
//! This convoluted mechanism enables such macros to run on stable Rust 1.15,
//! even though functional procedural macros (a.k.a. *Macros 2.0*) are not available yet.
//!
//! A library defining such a macro needs two crates: a “normal” one, and a `proc-macro` one.
//! In the example below we’ll call them `libfoo` and `libfoo-macros`, respectively.
//!
//! # Credits
//!
//! The trick that makes this crate work
//! is based on an idea from [David Tolnay](https://github.com/dtolnay).
//! Many thanks!
//!
//! # Example
//!
//! As a simple example, we’re going to re-implement the `stringify!` macro.
//! This is useless since `stringify!` already exists in the standard library,
//! and a bit absurd since this crate uses `stringify!` internally.
//!
//! Nevertheless, it serves as a simple example to demonstrate the use of this crate.
//!
//! ## The `proc-macro` crate
//!
//! The minimal `Cargo.toml` file is typical for Macros 1.1:
//!
//! ```toml
//! [package]
//! name = "libfoo-macros"
//! version = "1.0.0"
//!
//! [lib]
//! proc-macro = true
//! ```
//!
//! In the code, we define the procedural part of our macro in a function.
//! This function will not be used directly by end users,
//! but it still needs to be re-exported to them
//! (because of limitations in `macro_rules!`).
//!
//! To avoid name collisions, we include a long and explicit prefix in the function’s name.
//!
//! The function takes a string containing arbitrary Rust tokens,
//! and returns a string that is parsed as *items*.
//! The returned string can contain constants, statics, functions, `impl`s, etc.,
//! but not expressions directly.
//!
//! ```rust
//! #[macro_use] extern crate procedural_masquerade;
//! extern crate proc_macro;
//!
//! define_proc_macros! {
//!     #[allow(non_snake_case)]
//!     pub fn foo_internal__stringify_const(input: &str) -> String {
//!         format!("const STRINGIFIED: &'static str = {:?};", input)
//!     }
//! }
//! ```
//!
//! A less trivial macro would probably use
//! the [`syn`](https://github.com/dtolnay/syn/) crate to parse its input
//! and the [`quote`](https://github.com/dtolnay/quote) crate to generate its output.
//!
//! ## The library crate
//!
//! ```toml
//! [package]
//! name = "libfoo"
//! version = "1.0.0"
//!
//! [dependencies]
//! libfoo-macros = {path = "./macros", version = "1.0"}
//! ```
//!
//! ```rust
//! #[macro_use] extern crate libfoo_macros;  // (1)
//!
//! pub use libfoo_macros::*;  // (2)
//!
//! define_invoke_proc_macro!(libfoo__invoke_proc_macro);  // (3)
//!
//! #[macro_export]
//! macro_rules! foo_stringify {  // (4)
//!     ( $( $tts: tt ) ) => {
//!         {  // (5)
//!             libfoo__invoke_proc_macro! {  // (6)
//!                 foo_internal__stringify_const!( $( $tts ) )  // (7)
//!             }
//!             STRINGIFIED  // (8)
//!         }
//!     }
//! }
//! ```
//!
//! Let’s go trough the numbered lines one by one:
//!
//! 1. `libfoo` depends on `libfoo-macros`, and imports its macros.
//! 2. Everything exported by `libfoo-macros` (which is one custom `derive`)
//!    is re-exported to users of `libfoo`.
//!    They’re not expected to use it directly,
//!    but expansion of the `foo_stringify` macro needs it.
//! 3. This macro invocation defines yet another macro, called `libfoo__invoke_proc_macro`,
//!    which is also exported.
//!    This indirection is necessary
//!    because re-exporting `macro_rules!` macros doesn’t work currently,
//!    and once again it is used by the expansion of `foo_stringify`.
//!    Again, we use a long prefix to avoid name collisions.
//! 4. Finally, we define the macro that we really want.
//!    This one has a name that users will use.
//! 5. The expansion of this macro will define some items,
//!    whose names are not hygienic in `macro_rules`.
//!    So we wrap everything in an extra `{…}` block to prevent these names for leaking.
//! 6. Here we use the macro defined in (3),
//!    which allows us to write something that look like invoking a functional procedural macro,
//!    but really uses a custom `derive`.
//!    This will define a type called `ProceduralMasqueradeDummyType`,
//!    as a placeholder to use `derive`.
//!    If `libfoo__invoke_proc_macro!` is to be used more than once,
//!    each use needs to be nested in another block
//!    so that the names of multiple dummy types don’t collide.
//! 7. In addition to the dummy type,
//!    the items returned by our procedural component are inserted here.
//!    (In this case the `STRINGIFIED` constant.)
//! 8. Finally, we write the expression that we want the macro to evaluate to.
//!    This expression can use parts of `foo_stringify`’s input,
//!    it can contain control-flow statements like `return` or `continue`,
//!    and of course refer to procedurally-defined items.
//!
//! This macro can be used in an expression context.
//! It expands to a block-expression that contains some items (as an implementation detail)
//! and ends with another expression.
//!
//! ## For users
//!
//! Users of `libfoo` don’t need to worry about any of these implementation details.
//! They can use the `foo_stringify` macro as if it were a simle `macro_rules` macro:
//!
//! ```rust
//! #[macro_use] extern crate libfoo;
//!
//! fn main() {
//!     do_something(foo_stringify!(1 + 2));
//! }
//!
//! fn do_something(_: &str) { /* ... */ }
//! ```
//!
//! # More
//!
//! To see a more complex example, look at
//! [`cssparser`’s `src/macros.rs`](https://github.com/servo/rust-cssparser/blob/master/src/macros.rs)
//! and
//! [`cssparser-macros`’s `macros/lib.rs`](https://github.com/servo/rust-cssparser/blob/master/macros/lib.rs).

/// This macro wraps `&str -> String` functions
/// in custom `derive` implementations with `#[proc_macro_derive]`.
///
/// See crate documentation for details.
#[macro_export]
macro_rules! define_proc_macros {
    (
        $(
            $( #[$attr:meta] )*
            pub fn $proc_macro_name: ident ($input: ident : &str) -> String
            $body: block
        )+
    ) => {
        $(
            $( #[$attr] )*
            #[proc_macro_derive($proc_macro_name)]
            pub fn $proc_macro_name(derive_input: ::proc_macro::TokenStream)
                                    -> ::proc_macro::TokenStream {
                fn wrapped($input: &str) -> String {
                    $body
                }

                // syn uses a huge amount of stack in debug mode.
                let derive_input_string = derive_input.to_string();
                let handle =
                    ::std::thread::Builder::new().stack_size(128 * 1024 * 1024).spawn(move || {
                        wrapped($crate::_extract_input(&derive_input_string))
                    }).unwrap();

                handle.join().unwrap().parse().unwrap()
            }
        )+
    }
}

/// Implementation detail of `define_proc_macros!`.
///
/// **This function is not part of the public API. It can change or be removed between any versions.**
#[doc(hidden)]
pub fn _extract_input(derive_input: &str) -> &str {
    let mut input = derive_input;

    for expected in &[
        "#",
        "[",
        "allow",
        "(",
        "unused",
        ")",
        "]",
        "enum",
        "ProceduralMasqueradeDummyType",
        "{",
        "Input",
        "=",
        "(",
        "0",
        ",",
        "stringify",
        "!",
        "(",
    ] {
        input = input.trim_start();
        assert!(
            input.starts_with(expected),
            "expected prefix {:?} not found in {:?}",
            expected,
            derive_input
        );
        input = &input[expected.len()..];
    }

    for expected in [")", ")", ".", "0", ",", "}"].iter().rev() {
        input = input.trim_end();
        assert!(
            input.ends_with(expected),
            "expected suffix {:?} not found in {:?}",
            expected,
            derive_input
        );
        let end = input.len() - expected.len();
        input = &input[..end];
    }

    input
}

/// This macro expands to the definition of another macro (whose name is given as a parameter).
///
/// See crate documentation for details.
#[macro_export]
macro_rules! define_invoke_proc_macro {
    ($macro_name: ident) => {
        /// Implementation detail of other macros in this crate.
        #[doc(hidden)]
        #[macro_export]
        macro_rules! $macro_name {
                            ($proc_macro_name: ident ! $paren: tt) => {
                                #[derive($proc_macro_name)]
                                #[allow(unused)]
                                enum ProceduralMasqueradeDummyType {
                                    // The magic happens here.
                                    //
                                    // We use an `enum` with an explicit discriminant
                                    // because that is the only case where a type definition
                                    // can contain a (const) expression.
                                    //
                                    // `(0, "foo").0` evalutes to 0, with the `"foo"` part ignored.
                                    //
                                    // By the time the `#[proc_macro_derive]` function
                                    // implementing `#[derive($proc_macro_name)]` is called,
                                    // `$paren` has already been replaced with the input of this inner macro,
                                    // but `stringify!` has not been expanded yet.
                                    //
                                    // This how arbitrary tokens can be inserted
                                    // in the input to the `#[proc_macro_derive]` function.
                                    //
                                    // Later, `stringify!(...)` is expanded into a string literal
                                    // which is then ignored.
                                    // Using `stringify!` enables passing arbitrary tokens
                                    // rather than only what can be parsed as a const expression.
                                    Input = (0, stringify! $paren ).0,
                                }
                            }
                        }
    };
}
<commit_msg>Add rustfmt::skip to skip rustfmt for ProceduralMasqueradeDummyType enum<commit_after>//! # Custom `derive` pretending to be functional procedural macros on Rust 1.15
//!
//! This crate enables creating function-like macros (invoked as `foo!(...)`)
//! with a procedural component,
//! based on both custom `derive` (a.k.a. *Macros 1.1*) and `macro_rules!`.
//!
//! This convoluted mechanism enables such macros to run on stable Rust 1.15,
//! even though functional procedural macros (a.k.a. *Macros 2.0*) are not available yet.
//!
//! A library defining such a macro needs two crates: a “normal” one, and a `proc-macro` one.
//! In the example below we’ll call them `libfoo` and `libfoo-macros`, respectively.
//!
//! # Credits
//!
//! The trick that makes this crate work
//! is based on an idea from [David Tolnay](https://github.com/dtolnay).
//! Many thanks!
//!
//! # Example
//!
//! As a simple example, we’re going to re-implement the `stringify!` macro.
//! This is useless since `stringify!` already exists in the standard library,
//! and a bit absurd since this crate uses `stringify!` internally.
//!
//! Nevertheless, it serves as a simple example to demonstrate the use of this crate.
//!
//! ## The `proc-macro` crate
//!
//! The minimal `Cargo.toml` file is typical for Macros 1.1:
//!
//! ```toml
//! [package]
//! name = "libfoo-macros"
//! version = "1.0.0"
//!
//! [lib]
//! proc-macro = true
//! ```
//!
//! In the code, we define the procedural part of our macro in a function.
//! This function will not be used directly by end users,
//! but it still needs to be re-exported to them
//! (because of limitations in `macro_rules!`).
//!
//! To avoid name collisions, we include a long and explicit prefix in the function’s name.
//!
//! The function takes a string containing arbitrary Rust tokens,
//! and returns a string that is parsed as *items*.
//! The returned string can contain constants, statics, functions, `impl`s, etc.,
//! but not expressions directly.
//!
//! ```rust
//! #[macro_use] extern crate procedural_masquerade;
//! extern crate proc_macro;
//!
//! define_proc_macros! {
//!     #[allow(non_snake_case)]
//!     pub fn foo_internal__stringify_const(input: &str) -> String {
//!         format!("const STRINGIFIED: &'static str = {:?};", input)
//!     }
//! }
//! ```
//!
//! A less trivial macro would probably use
//! the [`syn`](https://github.com/dtolnay/syn/) crate to parse its input
//! and the [`quote`](https://github.com/dtolnay/quote) crate to generate its output.
//!
//! ## The library crate
//!
//! ```toml
//! [package]
//! name = "libfoo"
//! version = "1.0.0"
//!
//! [dependencies]
//! libfoo-macros = {path = "./macros", version = "1.0"}
//! ```
//!
//! ```rust
//! #[macro_use] extern crate libfoo_macros;  // (1)
//!
//! pub use libfoo_macros::*;  // (2)
//!
//! define_invoke_proc_macro!(libfoo__invoke_proc_macro);  // (3)
//!
//! #[macro_export]
//! macro_rules! foo_stringify {  // (4)
//!     ( $( $tts: tt ) ) => {
//!         {  // (5)
//!             libfoo__invoke_proc_macro! {  // (6)
//!                 foo_internal__stringify_const!( $( $tts ) )  // (7)
//!             }
//!             STRINGIFIED  // (8)
//!         }
//!     }
//! }
//! ```
//!
//! Let’s go trough the numbered lines one by one:
//!
//! 1. `libfoo` depends on `libfoo-macros`, and imports its macros.
//! 2. Everything exported by `libfoo-macros` (which is one custom `derive`)
//!    is re-exported to users of `libfoo`.
//!    They’re not expected to use it directly,
//!    but expansion of the `foo_stringify` macro needs it.
//! 3. This macro invocation defines yet another macro, called `libfoo__invoke_proc_macro`,
//!    which is also exported.
//!    This indirection is necessary
//!    because re-exporting `macro_rules!` macros doesn’t work currently,
//!    and once again it is used by the expansion of `foo_stringify`.
//!    Again, we use a long prefix to avoid name collisions.
//! 4. Finally, we define the macro that we really want.
//!    This one has a name that users will use.
//! 5. The expansion of this macro will define some items,
//!    whose names are not hygienic in `macro_rules`.
//!    So we wrap everything in an extra `{…}` block to prevent these names for leaking.
//! 6. Here we use the macro defined in (3),
//!    which allows us to write something that look like invoking a functional procedural macro,
//!    but really uses a custom `derive`.
//!    This will define a type called `ProceduralMasqueradeDummyType`,
//!    as a placeholder to use `derive`.
//!    If `libfoo__invoke_proc_macro!` is to be used more than once,
//!    each use needs to be nested in another block
//!    so that the names of multiple dummy types don’t collide.
//! 7. In addition to the dummy type,
//!    the items returned by our procedural component are inserted here.
//!    (In this case the `STRINGIFIED` constant.)
//! 8. Finally, we write the expression that we want the macro to evaluate to.
//!    This expression can use parts of `foo_stringify`’s input,
//!    it can contain control-flow statements like `return` or `continue`,
//!    and of course refer to procedurally-defined items.
//!
//! This macro can be used in an expression context.
//! It expands to a block-expression that contains some items (as an implementation detail)
//! and ends with another expression.
//!
//! ## For users
//!
//! Users of `libfoo` don’t need to worry about any of these implementation details.
//! They can use the `foo_stringify` macro as if it were a simle `macro_rules` macro:
//!
//! ```rust
//! #[macro_use] extern crate libfoo;
//!
//! fn main() {
//!     do_something(foo_stringify!(1 + 2));
//! }
//!
//! fn do_something(_: &str) { /* ... */ }
//! ```
//!
//! # More
//!
//! To see a more complex example, look at
//! [`cssparser`’s `src/macros.rs`](https://github.com/servo/rust-cssparser/blob/master/src/macros.rs)
//! and
//! [`cssparser-macros`’s `macros/lib.rs`](https://github.com/servo/rust-cssparser/blob/master/macros/lib.rs).

/// This macro wraps `&str -> String` functions
/// in custom `derive` implementations with `#[proc_macro_derive]`.
///
/// See crate documentation for details.
#[macro_export]
macro_rules! define_proc_macros {
    (
        $(
            $( #[$attr:meta] )*
            pub fn $proc_macro_name: ident ($input: ident : &str) -> String
            $body: block
        )+
    ) => {
        $(
            $( #[$attr] )*
            #[proc_macro_derive($proc_macro_name)]
            pub fn $proc_macro_name(derive_input: ::proc_macro::TokenStream)
                                    -> ::proc_macro::TokenStream {
                fn wrapped($input: &str) -> String {
                    $body
                }

                // syn uses a huge amount of stack in debug mode.
                let derive_input_string = derive_input.to_string();
                let handle =
                    ::std::thread::Builder::new().stack_size(128 * 1024 * 1024).spawn(move || {
                        wrapped($crate::_extract_input(&derive_input_string))
                    }).unwrap();

                handle.join().unwrap().parse().unwrap()
            }
        )+
    }
}

/// Implementation detail of `define_proc_macros!`.
///
/// **This function is not part of the public API. It can change or be removed between any versions.**
#[doc(hidden)]
pub fn _extract_input(derive_input: &str) -> &str {
    let mut input = derive_input;

    for expected in &[
        "#",
        "[",
        "allow",
        "(",
        "unused",
        ")",
        "]",
        "enum",
        "ProceduralMasqueradeDummyType",
        "{",
        "Input",
        "=",
        "(",
        "0",
        ",",
        "stringify",
        "!",
        "(",
    ] {
        input = input.trim_start();
        assert!(
            input.starts_with(expected),
            "expected prefix {:?} not found in {:?}",
            expected,
            derive_input
        );
        input = &input[expected.len()..];
    }

    for expected in [")", ")", ".", "0", ",", "}"].iter().rev() {
        input = input.trim_end();
        assert!(
            input.ends_with(expected),
            "expected suffix {:?} not found in {:?}",
            expected,
            derive_input
        );
        let end = input.len() - expected.len();
        input = &input[..end];
    }

    input
}

/// This macro expands to the definition of another macro (whose name is given as a parameter).
///
/// See crate documentation for details.
#[macro_export]
macro_rules! define_invoke_proc_macro {
    ($macro_name: ident) => {
        /// Implementation detail of other macros in this crate.
        #![rustfmt::skip]
        #[doc(hidden)]
        #[macro_export]
        macro_rules! $macro_name {
            ($proc_macro_name: ident ! $paren: tt) => {
                #[derive($proc_macro_name)]
                #[allow(unused)]
                enum ProceduralMasqueradeDummyType {
                    // The magic happens here.
                    //
                    // We use an `enum` with an explicit discriminant
                    // because that is the only case where a type definition
                    // can contain a (const) expression.
                    //
                    // `(0, "foo").0` evalutes to 0, with the `"foo"` part ignored.
                    //
                    // By the time the `#[proc_macro_derive]` function
                    // implementing `#[derive($proc_macro_name)]` is called,
                    // `$paren` has already been replaced with the input of this inner macro,
                    // but `stringify!` has not been expanded yet.
                    //
                    // This how arbitrary tokens can be inserted
                    // in the input to the `#[proc_macro_derive]` function.
                    //
                    // Later, `stringify!(...)` is expanded into a string literal
                    // which is then ignored.
                    // Using `stringify!` enables passing arbitrary tokens
                    // rather than only what can be parsed as a const expression.
                    Input = (0, stringify! $paren ).0,
                }
            }
        }
    };
}
<|endoftext|>


<commit_before>extern crate inkwell;

use self::inkwell::context::Context;
use self::inkwell::targets::{
    CodeModel, FileType, InitializationConfig, RelocMode, Target, TargetMachine,
};
use self::inkwell::values::BasicValue;
use self::inkwell::OptimizationLevel;

fn get_native_target_machine() -> TargetMachine {
    Target::initialize_native(&InitializationConfig::default())
        .expect("Failed to initialize native target");
    let target_triple = TargetMachine::get_default_triple();
    let target = Target::from_triple(&target_triple).unwrap();
    target
        .create_target_machine(
            &target_triple,
            &TargetMachine::get_host_cpu_name().to_string(),
            &TargetMachine::get_host_cpu_features().to_string(),
            OptimizationLevel::None,
            RelocMode::Static,
            CodeModel::Small,
        )
        .unwrap()
}

#[test]
fn test_section_iterator() {
    let target_machine = get_native_target_machine();

    let context = Context::create();
    let mut module = context.create_module("test_section_iterator");

    let gv_a = module.add_global(context.i8_type(), None, "a");
    gv_a.set_initializer(&context.i8_type().const_zero().as_basic_value_enum());
    gv_a.set_section("A");

    let gv_b = module.add_global(context.i16_type(), None, "b");
    gv_b.set_initializer(&context.i16_type().const_zero().as_basic_value_enum());
    gv_b.set_section("B");

    let gv_c = module.add_global(context.i32_type(), None, "c");
    gv_c.set_initializer(&context.i32_type().const_zero().as_basic_value_enum());
    gv_c.set_section("C");

    module.set_triple(&target_machine.get_triple());
    module.set_data_layout(&target_machine.get_target_data().get_data_layout());

    let memory_buffer = target_machine
        .write_to_memory_buffer(&mut module, FileType::Object)
        .unwrap();
    let object_file = memory_buffer.create_object_file().unwrap();

    let mut has_section_a = false;
    let mut has_section_b = false;
    let mut has_section_c = false;
    for section in object_file.get_sections() {
        if let Some(name) = section.get_name() {
            match name.to_str().unwrap() {
                "A" => {
                    assert!(!has_section_a);
                    has_section_a = true;
                    assert_eq!(section.size(), 1);
                }
                "B" => {
                    assert!(!has_section_b);
                    has_section_b = true;
                    assert_eq!(section.size(), 2);
                }
                "C" => {
                    assert!(!has_section_c);
                    has_section_c = true;
                    assert_eq!(section.size(), 4);
                }
                _ => {}
            }
        }
    }
    assert!(has_section_a);
    assert!(has_section_b);
    assert!(has_section_c);
}

#[test]
fn test_symbol_iterator() {
    let target_machine = get_native_target_machine();

    let context = Context::create();
    let mut module = context.create_module("test_symbol_iterator");
    module
        .add_global(context.i8_type(), None, "a")
        .set_initializer(&context.i8_type().const_zero().as_basic_value_enum());
    module
        .add_global(context.i16_type(), None, "b")
        .set_initializer(&context.i16_type().const_zero().as_basic_value_enum());
    module
        .add_global(context.i32_type(), None, "c")
        .set_initializer(&context.i32_type().const_zero().as_basic_value_enum());
    module.set_triple(&target_machine.get_triple());
    module.set_data_layout(&target_machine.get_target_data().get_data_layout());

    let memory_buffer = target_machine
        .write_to_memory_buffer(&mut module, FileType::Object)
        .unwrap();
    let object_file = memory_buffer.create_object_file().unwrap();

    let mut has_symbol_a = false;
    let mut has_symbol_b = false;
    let mut has_symbol_c = false;
    for symbol in object_file.get_symbols() {
        if let Some(name) = symbol.get_name() {
            match name.to_str().unwrap() {
                "a" => {
                    assert!(!has_symbol_a);
                    has_symbol_a = true;
                    assert_eq!(symbol.size(), 1);
                }
                "b" => {
                    assert!(!has_symbol_b);
                    has_symbol_b = true;
                    assert_eq!(symbol.size(), 2);
                }
                "c" => {
                    assert!(!has_symbol_c);
                    has_symbol_c = true;
                    assert_eq!(symbol.size(), 4);
                }
                _ => {}
            }
        }
    }
    assert!(has_symbol_a);
    assert!(has_symbol_b);
    assert!(has_symbol_c);
}

#[test]
fn test_reloc_iterator() {
    let target_machine = get_native_target_machine();

    let context = Context::create();
    let target_data = target_machine.get_target_data();
    let intptr_t = target_data.ptr_sized_int_type_in_context(&context, None);

    let mut module = context.create_module("test_reloc_iterator");
    let x_ptr = module
        .add_global(context.i8_type(), None, "x")
        .as_pointer_value();
    let x_plus_4 = x_ptr
        .const_to_int(intptr_t)
        .const_add(intptr_t.const_int(4, false));
    module
        .add_global(intptr_t, None, "a")
        .set_initializer(&x_plus_4);

    module.set_triple(&target_machine.get_triple());
    module.set_data_layout(&target_machine.get_target_data().get_data_layout());

    let memory_buffer = target_machine
        .write_to_memory_buffer(&mut module, FileType::Object)
        .unwrap();
    let object_file = memory_buffer.create_object_file().unwrap();

    let mut found_relocation = false;
    for section in object_file.get_sections() {
        for _ in section.get_relocations() {
            found_relocation = true;
            // We don't stop the traversal here, so as to exercise the iterators.
        }
    }
    assert!(found_relocation);
}
<commit_msg>An attempt at fixing the tests for LLVM 3.x-series.<commit_after>extern crate inkwell;

use self::inkwell::context::Context;
use self::inkwell::module::Module;
use self::inkwell::targets::{
    CodeModel, FileType, InitializationConfig, RelocMode, Target, TargetMachine,
};
use self::inkwell::types::IntType;
use self::inkwell::values::BasicValue;
use self::inkwell::OptimizationLevel;

#[llvm_versions(4.0..=latest)]
fn get_host_cpu_name() -> String {
    TargetMachine::get_host_cpu_name().to_string()
}
#[llvm_versions(4.0..=latest)]
fn get_host_cpu_features() -> String {
    TargetMachine::get_host_cpu_features().to_string()
}
#[llvm_versions(4.0..=latest)]
fn ptr_sized_int_type<'ctx>(
    target_machine: &TargetMachine,
    context: &'ctx Context,
) -> IntType<'ctx> {
    let target_data = target_machine.get_target_data();
    target_data.ptr_sized_int_type_in_context(&context, None)
}
#[llvm_versions(4.0..=latest)]
fn apply_target_to_module<'ctx>(target_machine: &TargetMachine, module: &Module) {
    module.set_triple(&target_machine.get_triple());
    module.set_data_layout(&target_machine.get_target_data().get_data_layout());
}

#[llvm_versions(3.6..4.0)]
fn get_host_cpu_name() -> String {
    ""
}
#[llvm_versions(3.6..4.0)]
fn get_host_cpu_features() -> String {
    ""
}
#[llvm_versions(3.6..4.0)]
fn ptr_sized_int_type<'ctx>(
    _target_machine: &TargetMachine,
    context: &'ctx Context,
) -> IntType<'ctx> {
    context.i64_type()
}
#[llvm_versions(3.6..4.0)]
fn apply_target_to_module(target_machine: &TargetMachine, module: &Module) {
    module.set_triple(&target_machine.get_triple());
}

fn get_native_target_machine() -> TargetMachine {
    Target::initialize_native(&InitializationConfig::default())
        .expect("Failed to initialize native target");
    let target_triple = TargetMachine::get_default_triple();
    let target = Target::from_triple(&target_triple).unwrap();
    target
        .create_target_machine(
            &target_triple,
            &get_host_cpu_name(),
            &get_host_cpu_features(),
            OptimizationLevel::None,
            RelocMode::Default,
            CodeModel::Default,
        )
        .unwrap()
}

#[test]
fn test_section_iterator() {
    let target_machine = get_native_target_machine();

    let context = Context::create();
    let mut module = context.create_module("test_section_iterator");

    let gv_a = module.add_global(context.i8_type(), None, "a");
    gv_a.set_initializer(&context.i8_type().const_zero().as_basic_value_enum());
    gv_a.set_section("A");

    let gv_b = module.add_global(context.i16_type(), None, "b");
    gv_b.set_initializer(&context.i16_type().const_zero().as_basic_value_enum());
    gv_b.set_section("B");

    let gv_c = module.add_global(context.i32_type(), None, "c");
    gv_c.set_initializer(&context.i32_type().const_zero().as_basic_value_enum());
    gv_c.set_section("C");

    apply_target_to_module(&target_machine, &module);

    let memory_buffer = target_machine
        .write_to_memory_buffer(&mut module, FileType::Object)
        .unwrap();
    let object_file = memory_buffer.create_object_file().unwrap();

    let mut has_section_a = false;
    let mut has_section_b = false;
    let mut has_section_c = false;
    for section in object_file.get_sections() {
        if let Some(name) = section.get_name() {
            match name.to_str().unwrap() {
                "A" => {
                    assert!(!has_section_a);
                    has_section_a = true;
                    assert_eq!(section.size(), 1);
                }
                "B" => {
                    assert!(!has_section_b);
                    has_section_b = true;
                    assert_eq!(section.size(), 2);
                }
                "C" => {
                    assert!(!has_section_c);
                    has_section_c = true;
                    assert_eq!(section.size(), 4);
                }
                _ => {}
            }
        }
    }
    assert!(has_section_a);
    assert!(has_section_b);
    assert!(has_section_c);
}

#[test]
fn test_symbol_iterator() {
    let target_machine = get_native_target_machine();

    let context = Context::create();
    let mut module = context.create_module("test_symbol_iterator");
    module
        .add_global(context.i8_type(), None, "a")
        .set_initializer(&context.i8_type().const_zero().as_basic_value_enum());
    module
        .add_global(context.i16_type(), None, "b")
        .set_initializer(&context.i16_type().const_zero().as_basic_value_enum());
    module
        .add_global(context.i32_type(), None, "c")
        .set_initializer(&context.i32_type().const_zero().as_basic_value_enum());
    apply_target_to_module(&target_machine, &module);

    let memory_buffer = target_machine
        .write_to_memory_buffer(&mut module, FileType::Object)
        .unwrap();
    let object_file = memory_buffer.create_object_file().unwrap();

    let mut has_symbol_a = false;
    let mut has_symbol_b = false;
    let mut has_symbol_c = false;
    for symbol in object_file.get_symbols() {
        if let Some(name) = symbol.get_name() {
            match name.to_str().unwrap() {
                "a" => {
                    assert!(!has_symbol_a);
                    has_symbol_a = true;
                    assert_eq!(symbol.size(), 1);
                }
                "b" => {
                    assert!(!has_symbol_b);
                    has_symbol_b = true;
                    assert_eq!(symbol.size(), 2);
                }
                "c" => {
                    assert!(!has_symbol_c);
                    has_symbol_c = true;
                    assert_eq!(symbol.size(), 4);
                }
                _ => {}
            }
        }
    }
    assert!(has_symbol_a);
    assert!(has_symbol_b);
    assert!(has_symbol_c);
}

#[test]
fn test_reloc_iterator() {
    let target_machine = get_native_target_machine();

    let context = Context::create();
    let intptr_t = ptr_sized_int_type(&target_machine, &context);

    let mut module = context.create_module("test_reloc_iterator");
    let x_ptr = module
        .add_global(context.i8_type(), None, "x")
        .as_pointer_value();
    let x_plus_4 = x_ptr
        .const_to_int(intptr_t)
        .const_add(intptr_t.const_int(4, false));
    module
        .add_global(intptr_t, None, "a")
        .set_initializer(&x_plus_4);

    apply_target_to_module(&target_machine, &module);

    let memory_buffer = target_machine
        .write_to_memory_buffer(&mut module, FileType::Object)
        .unwrap();
    let object_file = memory_buffer.create_object_file().unwrap();

    let mut found_relocation = false;
    for section in object_file.get_sections() {
        for _ in section.get_relocations() {
            found_relocation = true;
            // We don't stop the traversal here, so as to exercise the iterators.
        }
    }
    assert!(found_relocation);
}
<|endoftext|>


<commit_before>use crate::common::util::*;

#[test]
fn test_default_mode() {
    // test the default mode

    // accept some reasonable default
    new_ucmd!().args(&["abc/def"]).succeeds().no_stdout();

    // fail on long inputs
    new_ucmd!()
        .args(&["test".repeat(20000)])
        .fails()
        .no_stdout();
}
<commit_msg>pathchk: improve unit test code coverage with more tests<commit_after>use crate::common::util::*;

#[test]
fn test_default_mode() {
    // test the default mode

    // accept some reasonable default
    new_ucmd!().args(&["dir/file"]).succeeds().no_stdout();

    // accept non-portable chars
    new_ucmd!().args(&["dir#/$file"]).succeeds().no_stdout();

    // accept empty path
    new_ucmd!().args(&[""]).succeeds().no_stdout();

    // fail on long path
    new_ucmd!()
        .args(&["dir".repeat(libc::PATH_MAX as usize + 1)])
        .fails()
        .no_stdout();

    // fail on long filename
    new_ucmd!()
        .args(&[format!(
            "dir/{}",
            "file".repeat(libc::FILENAME_MAX as usize + 1)
        )])
        .fails()
        .no_stdout();
}

#[test]
fn test_posix_mode() {
    // test the posix mode

    // accept some reasonable default
    new_ucmd!().args(&["-p", "dir/file"]).succeeds().no_stdout();

    // fail on long path
    new_ucmd!()
        .args(&[
            "-p",
            &"dir".repeat(libc::PATH_MAX as usize + 1).as_str(),
        ])
        .fails()
        .no_stdout();

    // fail on long filename
    new_ucmd!()
        .args(&[
            "-p",
            &format!("dir/{}", "file".repeat(libc::FILENAME_MAX as usize + 1)).as_str(),
        ])
        .fails()
        .no_stdout();

    // fail on non-portable chars
    new_ucmd!().args(&["-p", "dir#/$file"]).fails().no_stdout();
}

#[test]
fn test_posix_special() {
    // test the posix special mode

    // accept some reasonable default
    new_ucmd!().args(&["-P", "dir/file"]).succeeds().no_stdout();

    // accept non-portable chars
    new_ucmd!()
        .args(&["-P", "dir#/$file"])
        .succeeds()
        .no_stdout();

    // accept non-leading hyphen
    new_ucmd!()
        .args(&["-P", "dir/file-name"])
        .succeeds()
        .no_stdout();

    // fail on long path
    new_ucmd!()
        .args(&[
            "-P",
            &"dir".repeat(libc::PATH_MAX as usize + 1).as_str(),
        ])
        .fails()
        .no_stdout();

    // fail on long filename
    new_ucmd!()
        .args(&[
            "-P",
            &format!("dir/{}", "file".repeat(libc::FILENAME_MAX as usize + 1)).as_str(),
        ])
        .fails()
        .no_stdout();

    // fail on leading hyphen char
    new_ucmd!().args(&["-P", "dir/-file"]).fails().no_stdout();

    // fail on empty path
    new_ucmd!().args(&["-P", ""]).fails().no_stdout();
}

#[test]
fn test_posix_all() {
    // test the posix special mode

    // accept some reasonable default
    new_ucmd!().args(&["-p", "-P", "dir/file"]).succeeds().no_stdout();

    // accept non-leading hyphen
    new_ucmd!()
        .args(&["-p", "-P", "dir/file-name"])
        .succeeds()
        .no_stdout();

    // fail on long path
    new_ucmd!()
        .args(&[
            "-p",
            "-P",
            &"dir".repeat(libc::PATH_MAX as usize + 1).as_str(),
        ])
        .fails()
        .no_stdout();

    // fail on long filename
    new_ucmd!()
        .args(&[
            "-p",
            "-P",
            &format!("dir/{}", "file".repeat(libc::FILENAME_MAX as usize + 1)).as_str(),
        ])
        .fails()
        .no_stdout();

    // fail on non-portable chars
    new_ucmd!().args(&["-p", "-P", "dir#/$file"]).fails().no_stdout();

    // fail on leading hyphen char
    new_ucmd!().args(&["-p", "-P", "dir/-file"]).fails().no_stdout();

    // fail on empty path
    new_ucmd!().args(&["-p", "-P", ""]).fails().no_stdout();
}

#[test]
fn test_args_parsing() {
    // fail on no args
    let empty_args: [String; 0] = [];
    new_ucmd!()
        .args(&empty_args)
        .fails()
        .no_stdout();
}<|endoftext|>


<commit_before>extern crate server;
extern crate byteorder;
extern crate rustc_serialize;
extern crate bincode;

use server::net;
//use bincode::rustc_serialize::{encode_into};
//use bincode::SizeLimit;

/// test of some functions and structs by net
fn main() {
    let mut vec = Vec::new();   // stream to write into

    let err = net::NetworkErrors::UnexpectedPkg("unexpected".into());

    //test if the message is sent
    let _ = net::send_error_packet(&mut vec, err.into());
    println!("{:?}", vec );
}
<commit_msg>Remove annoying conntest.rs<commit_after><|endoftext|>


<commit_before>//! Higher-level Rust constructs for http_parser

use http_parser;
use core::vec::raw::from_buf_raw;
use core::libc::{c_int, c_void, c_char, size_t};
use core::ptr::{null, to_unsafe_ptr};
use http_parser::{http_parser_settings, HTTP_RESPONSE};
use http_parser::bindgen::{http_parser_init, http_parser_execute};

pub type HttpCallback = @fn() -> bool;
pub type HttpDataCallback = @fn(data: ~[u8]) -> bool;

pub struct ParserCallbacks {
    on_message_begin: HttpCallback,
    on_url: HttpDataCallback,
    on_header_field: HttpDataCallback,
    on_header_value: HttpDataCallback,
    on_headers_complete: HttpCallback,
    on_body: HttpDataCallback,
    on_message_complete: HttpCallback
}

pub struct Parser {
    http_parser: http_parser::http_parser,
    settings: http_parser_settings
}

pub fn Parser() -> Parser {
    let http_parser = http_parser::struct_http_parser {
        _type_flags: 0,
        state: 0,
        header_state: 0,
        index: 0,
        nread: 0,
        content_length: 0,
        http_major: 0,
        http_minor: 0,
        status_code: 0,
        method: 0,
        http_errno_upgrade: 0,
        data: null()
    };

    unsafe {
        http_parser_init(&http_parser, HTTP_RESPONSE);
    }

    let settings = http_parser::struct_http_parser_settings {
        on_message_begin: on_message_begin,
        on_url: on_url,
        on_header_field: on_header_field,
        on_header_value: on_header_value,
        on_headers_complete: on_headers_complete,
        on_body: on_body,
        on_message_complete: on_message_complete
    };

    Parser {
        http_parser: http_parser,
        settings: settings
    }
}

pub impl Parser {
    fn execute(&mut self, data: &[u8], callbacks: &ParserCallbacks) -> uint {
        unsafe {
            self.http_parser.data = to_unsafe_ptr(callbacks) as *c_void;
            do vec::as_imm_buf(data) |buf, _| {
                http_parser_execute(&self.http_parser,
                                    &self.settings,
                                    buf as *c_char,
                                    data.len() as size_t) as uint
            }
        }
    }

    fn status_code(&self) -> uint {
        self.http_parser.status_code as uint
    }
}

fn callbacks(http_parser: *http_parser::http_parser) -> *ParserCallbacks {
    unsafe {
        fail_unless!((*http_parser).data.is_not_null());
        return (*http_parser).data as *ParserCallbacks;
    }
}

extern fn on_message_begin(http_parser: *http_parser::http_parser) -> c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_message_begin)()) as c_int
    }
}

extern fn on_url(http_parser: *http_parser::http_parser, at: *u8, length: size_t) -> c_int {
    unsafe {
        (!(((*callbacks(http_parser)).on_url)(from_buf_raw(at, length as uint)))) as c_int
    }
}

extern fn on_header_field(http_parser: *http_parser::http_parser, at: *u8, length: size_t) ->
        c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_header_field)(from_buf_raw(at, length as uint))) as c_int
    }
}

extern fn on_header_value(http_parser: *http_parser::http_parser, at: *u8, length: size_t) ->
        c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_header_value)(from_buf_raw(at, length as uint))) as c_int
    }
}

extern fn on_headers_complete(http_parser: *http_parser::http_parser) -> c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_headers_complete)()) as c_int
    }
}

extern fn on_body(http_parser: *http_parser::http_parser, at: *u8, length: size_t) -> c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_body)(from_buf_raw(at, length as uint))) as c_int
    }
}

extern fn on_message_complete(http_parser: *http_parser::http_parser) -> c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_message_complete)()) as c_int
    }
}
<commit_msg>Remove fail_unless!.<commit_after>//! Higher-level Rust constructs for http_parser

use http_parser;
use core::vec::raw::from_buf_raw;
use core::libc::{c_int, c_void, c_char, size_t};
use core::ptr::{null, to_unsafe_ptr};
use http_parser::{http_parser_settings, HTTP_RESPONSE};
use http_parser::bindgen::{http_parser_init, http_parser_execute};

pub type HttpCallback = @fn() -> bool;
pub type HttpDataCallback = @fn(data: ~[u8]) -> bool;

pub struct ParserCallbacks {
    on_message_begin: HttpCallback,
    on_url: HttpDataCallback,
    on_header_field: HttpDataCallback,
    on_header_value: HttpDataCallback,
    on_headers_complete: HttpCallback,
    on_body: HttpDataCallback,
    on_message_complete: HttpCallback
}

pub struct Parser {
    http_parser: http_parser::http_parser,
    settings: http_parser_settings
}

pub fn Parser() -> Parser {
    let http_parser = http_parser::struct_http_parser {
        _type_flags: 0,
        state: 0,
        header_state: 0,
        index: 0,
        nread: 0,
        content_length: 0,
        http_major: 0,
        http_minor: 0,
        status_code: 0,
        method: 0,
        http_errno_upgrade: 0,
        data: null()
    };

    unsafe {
        http_parser_init(&http_parser, HTTP_RESPONSE);
    }

    let settings = http_parser::struct_http_parser_settings {
        on_message_begin: on_message_begin,
        on_url: on_url,
        on_header_field: on_header_field,
        on_header_value: on_header_value,
        on_headers_complete: on_headers_complete,
        on_body: on_body,
        on_message_complete: on_message_complete
    };

    Parser {
        http_parser: http_parser,
        settings: settings
    }
}

pub impl Parser {
    fn execute(&mut self, data: &[u8], callbacks: &ParserCallbacks) -> uint {
        unsafe {
            self.http_parser.data = to_unsafe_ptr(callbacks) as *c_void;
            do vec::as_imm_buf(data) |buf, _| {
                http_parser_execute(&self.http_parser,
                                    &self.settings,
                                    buf as *c_char,
                                    data.len() as size_t) as uint
            }
        }
    }

    fn status_code(&self) -> uint {
        self.http_parser.status_code as uint
    }
}

fn callbacks(http_parser: *http_parser::http_parser) -> *ParserCallbacks {
    unsafe {
        assert!((*http_parser).data.is_not_null());
        return (*http_parser).data as *ParserCallbacks;
    }
}

extern fn on_message_begin(http_parser: *http_parser::http_parser) -> c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_message_begin)()) as c_int
    }
}

extern fn on_url(http_parser: *http_parser::http_parser, at: *u8, length: size_t) -> c_int {
    unsafe {
        (!(((*callbacks(http_parser)).on_url)(from_buf_raw(at, length as uint)))) as c_int
    }
}

extern fn on_header_field(http_parser: *http_parser::http_parser, at: *u8, length: size_t) ->
        c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_header_field)(from_buf_raw(at, length as uint))) as c_int
    }
}

extern fn on_header_value(http_parser: *http_parser::http_parser, at: *u8, length: size_t) ->
        c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_header_value)(from_buf_raw(at, length as uint))) as c_int
    }
}

extern fn on_headers_complete(http_parser: *http_parser::http_parser) -> c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_headers_complete)()) as c_int
    }
}

extern fn on_body(http_parser: *http_parser::http_parser, at: *u8, length: size_t) -> c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_body)(from_buf_raw(at, length as uint))) as c_int
    }
}

extern fn on_message_complete(http_parser: *http_parser::http_parser) -> c_int {
    unsafe {
        (!((*callbacks(http_parser)).on_message_complete)()) as c_int
    }
}
<|endoftext|>


<commit_before>use bifrost::utils::async_locks::{Mutex, RwLock, RwLockReadGuard, RwLockWriteGuard};
use bifrost::utils::fut_exec::wait;
use client::AsyncClient;
use core::borrow::BorrowMut;
use dovahkiin::types::custom_types::id::Id;
use dovahkiin::types::custom_types::map::Map;
use dovahkiin::types::type_id_of;
use dovahkiin::types::value::ToValue;
use futures::Future;
use index::btree::*;
use index::EntryKey;
use index::Slice;
use itertools::Itertools;
use owning_ref::{OwningHandle, OwningRef, RcRef};
use ram::cell::Cell;
use ram::schema::{Field, Schema};
use ram::types::*;
use std::cell::Ref;
use std::cell::RefCell;
use std::cell::RefMut;
use std::cell::UnsafeCell;
use std::collections::BTreeSet;
use std::collections::HashMap;
use std::marker::PhantomData;
use std::mem;
use std::ops::Deref;
use std::ops::DerefMut;
use std::rc::Rc;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use utils::lru_cache::LRUCache;

const PAGE_SCHEMA: &'static str = "NEB_BTREE_PAGE";
const KEYS_FIELD: &'static str = "keys";
const NEXT_FIELD: &'static str = "next";
const PREV_FIELD: &'static str = "prev";

lazy_static! {
    static ref KEYS_KEY_HASH: u64 = key_hash(KEYS_FIELD);
    static ref NEXT_PAGE_KEY_HASH: u64 = key_hash(NEXT_FIELD);
    static ref PREV_PAGE_KEY_HASH: u64 = key_hash(PREV_FIELD);
    static ref PAGE_SCHEMA_ID: u32 = key_hash(PAGE_SCHEMA) as u32;
}

pub struct ExtNode<KS, PS>
where
    KS: Slice<EntryKey> + Debug + 'static,
    PS: Slice<NodeCellRef> + 'static,
{
    pub id: Id,
    pub keys: KS,
    pub next: NodeCellRef,
    pub prev: NodeCellRef,
    pub len: usize,
    pub dirty: bool,
    pub right_bound: EntryKey,
    mark: PhantomData<PS>,
}

pub struct ExtNodeSplit<KS, PS>
where
    KS: Slice<EntryKey> + Debug + 'static,
    PS: Slice<NodeCellRef> + 'static,
{
    pub node_2: ExtNode<KS, PS>,
    pub keys_1_len: usize,
}

impl<KS, PS> ExtNode<KS, PS>
where
    KS: Slice<EntryKey> + Debug + 'static,
    PS: Slice<NodeCellRef> + 'static,
{
    pub fn new(id: Id, right_bound: EntryKey) -> Box<Self> {
        box ExtNode {
            id,
            keys: KS::init(),
            next: Node::<KS, PS>::none_ref(),
            prev: Node::<KS, PS>::none_ref(),
            len: 0,
            dirty: false,
            right_bound,
            mark: PhantomData,
        }
    }

    pub fn to_cell(&self) -> Cell {
        let mut value = Value::Map(Map::new());
        let prev_id = read_node(&self.prev, |node: &NodeReadHandler<KS, PS>| {
            node.extnode().id
        });
        let next_id = read_node(&self.next, |node: &NodeReadHandler<KS, PS>| {
            node.extnode().id
        });

        value[*NEXT_PAGE_KEY_HASH] = Value::Id(prev_id);
        value[*PREV_PAGE_KEY_HASH] = Value::Id(next_id);
        value[*KEYS_KEY_HASH] = self.keys.as_slice_immute()[..self.len]
            .iter()
            .map(|key| SmallBytes::from_vec(key.as_slice().to_vec()))
            .collect_vec()
            .value();
        Cell::new_with_id(*PAGE_SCHEMA_ID, &self.id, value)
    }
    pub fn search(&self, key: &EntryKey) -> usize {
        self.keys.as_slice_immute()[..self.len]
            .binary_search(key)
            .unwrap_or_else(|i| i)
    }
    pub fn remove_at(&mut self, pos: usize) {
        let mut cached_len = &mut self.len;
        debug!(
            "Removing from external pos {}, len {}, key {:?}",
            pos,
            cached_len,
            self.keys.as_slice()[pos]
        );
        self.keys.remove_at(pos, cached_len);
    }

    pub fn split_insert(
        &mut self,
        key: EntryKey,
        pos: usize,
        self_ref: &NodeCellRef,
        self_next: &mut NodeWriteGuard<KS, PS>,
        tree: &BPlusTree<KS, PS>,
    ) -> (NodeCellRef, EntryKey) {
        // cached.dump();
        let pivot = self.len / 2;
        let new_page_id = tree.new_page_id();
        let mut keys_1 = &mut self.keys;
        let mut keys_2 = keys_1.split_at_pivot(pivot, self.len);
        let mut keys_1_len = pivot;
        let mut keys_2_len = self.len - pivot;
        // modify next node point previous to new node
        insert_into_split(
            key,
            keys_1,
            &mut keys_2,
            &mut keys_1_len,
            &mut keys_2_len,
            pos,
        );
        let pivot_key = keys_2.as_slice()[0].clone();
        let extnode_2: Box<ExtNode<KS, PS>> = box ExtNode {
            id: new_page_id,
            keys: keys_2,
            next: self.next.clone(),
            prev: self_ref.clone(),
            len: keys_2_len,
            dirty: true,
            right_bound: mem::replace(&mut self.right_bound, pivot_key.clone()),
            mark: PhantomData,
        };
        debug_assert!(pivot_key > smallvec!(0));
        debug_assert!(
            &pivot_key > &keys_1.as_slice()[keys_1_len - 1],
            "{:?} / {:?} @ {}",
            pivot_key,
            &keys_1.as_slice()[keys_1_len - 1],
            pos
        );
        debug_assert!(read_unchecked::<KS, PS>(&extnode_2.prev).is_ext());
        self.len = keys_1_len;
        debug!(
            "Split to left len {}, right len {}, right prev id: {:?}",
            self.len,
            extnode_2.len,
            read_unchecked::<KS, PS>(&extnode_2.prev).ext_id()
        );
        let node_2 = NodeCellRef::new(Node::with_external(extnode_2));
        {
            if !self_next.is_none() {
                self_next.extnode_mut().prev = node_2.clone();
            }
        }
        self.next = node_2.clone();

        (node_2, pivot_key)
    }

    pub fn insert(
        &mut self,
        key: &EntryKey,
        tree: &BPlusTree<KS, PS>,
        self_ref: &NodeCellRef,
        parent: &NodeCellRef,
    ) -> Option<NodeSplit<KS, PS>> {
        let key = key.clone();
        let pos = self.search(&key);
        debug_assert!(self.len <= KS::slice_len());
        debug_assert!(pos <= self.len);
        if self.len == KS::slice_len() {
            // need to split
            debug!("insert to external with split, key {:?}, pos {}", key, pos);
            let mut self_next: NodeWriteGuard<KS, PS> = write_node(&self.next);
            let mut parent_latch: NodeWriteGuard<KS, PS> = write_node(parent);
            let (node_2, pivot_key) = self.split_insert(key, pos, self_ref, &mut self_next, tree);
            Some(NodeSplit {
                new_right_node: node_2,
                pivot: pivot_key,
                parent_latch,
                left_node_latch: NodeWriteGuard::default(),
            })
        } else {
            debug!("insert to external without split at {}, key {:?}", pos, key);
            self.keys.insert_at(key, pos, &mut self.len);
            None
        }
    }

    pub fn merge_with(&mut self, right: &mut Self) {
        debug!(
            "Merge external node, left len {}:{:?}, right len {}:{:?}",
            self.len, self.keys, right.len, right.keys
        );
        let self_len = self.len;
        let new_len = self.len + right.len;
        debug_assert!(new_len <= KS::slice_len());
        for i in self.len..new_len {
            mem::swap(
                &mut self.keys.as_slice()[i],
                &mut right.keys.as_slice()[i - self_len],
            );
        }
        self.len = new_len;
    }

    pub fn remove_contains(&mut self, set: &mut BTreeSet<EntryKey>) {
        let remaining_keys = {
            let remaining = self.keys.as_slice()[..self.len]
                .iter()
                .filter(|&k| !set.remove(k))
                .collect_vec();
            if remaining.len() == self.len {
                return;
            }
            remaining.into_iter().cloned().collect_vec()
        };
        let mut self_key_slice = self.keys.as_slice();
        self.len = remaining_keys.len();
        for (i, k_ref) in remaining_keys.into_iter().enumerate() {
            self_key_slice[i] = k_ref;
        }
    }

    pub fn merge_sort(&mut self, right: &[&EntryKey]) {
        debug!("Merge sort have right nodes {:?}", right);
        let self_len_before_merge = self.len;
        debug_assert!(self_len_before_merge + right.len() <= KS::slice_len());
        let mut pos = 0;
        let mut left_pos = 0;
        let mut right_pos = 0;
        let mut left_keys = mem::replace(&mut self.keys, KS::init());
        let mut left = left_keys.as_slice();
        while left_pos < self.len && right_pos < right.len() {
            let left_key = &left[left_pos];
            let right_key = right[right_pos];
            if left_key < right_key {
                self.keys.as_slice()[pos] = left_key.clone();
                left_pos += 1;
            } else {
                self.keys.as_slice()[pos] = right_key.clone();
                right_pos += 1;
            }
            pos += 1;
        }
        for key in &left[left_pos..self.len] {
            self.keys.as_slice()[pos] = key.clone();
            pos += 1;
            left_pos += 1;
        }
        for key in &right[right_pos..] {
            self.keys.as_slice()[pos] = (*key).clone();
            pos += 1;
            right_pos += 1;
        }
        debug!(
            "Merge sorted have keys {:?}",
            &self.keys.as_slice_immute()[..pos]
        );
        self.len = pos;
        self.dirty = true;
        debug_assert_eq!(self_len_before_merge, left_pos);
        debug_assert_eq!(right.len(), right_pos);
        debug_assert_eq!(self.len, self_len_before_merge + right.len());
        debug!(
            "Merge sorted page have keys: {:?}",
            &self.keys.as_slice_immute()[..self.len]
        )
    }

    pub fn dump(&self) {
        debug!("Dumping {:?}, keys {}", self.id, self.len);
        for i in 0..KS::slice_len() {
            debug!("{}\t- {:?}", i, self.keys.as_slice_immute()[i]);
        }
    }
    pub fn is_dirty(&self) -> bool {
        unimplemented!()
    }
}

pub fn page_schema() -> Schema {
    Schema {
        id: *PAGE_SCHEMA_ID,
        name: String::from(PAGE_SCHEMA),
        key_field: None,
        str_key_field: None,
        is_dynamic: false,
        fields: Field::new(
            "*",
            0,
            false,
            false,
            Some(vec![
                Field::new(NEXT_FIELD, type_id_of(Type::Id), false, false, None),
                Field::new(PREV_FIELD, type_id_of(Type::Id), false, false, None),
                Field::new(KEYS_FIELD, type_id_of(Type::SmallBytes), false, true, None),
            ]),
        ),
    }
}

pub struct NodeCache<KS, PS>
where
    KS: Slice<EntryKey> + Debug + 'static,
    PS: Slice<NodeCellRef> + 'static,
{
    nodes: RefCell<HashMap<Id, NodeCellRef>>,
    storage: Arc<AsyncClient>,
    marker: PhantomData<(KS, PS)>,
}

impl<KS, PS> NodeCache<KS, PS>
where
    KS: Slice<EntryKey> + Debug + 'static,
    PS: Slice<NodeCellRef> + 'static,
{
    pub fn new(neb_client: &Arc<AsyncClient>) -> Self {
        NodeCache {
            nodes: RefCell::new(HashMap::new()),
            storage: neb_client.clone(),
            marker: PhantomData,
        }
    }

    pub fn get(&self, id: &Id) -> NodeCellRef {
        if id.is_unit_id() {
            return NodeCellRef::new(Node::<KS, PS>::with_none());
        }
        let mut nodes = self.nodes.borrow_mut();
        nodes
            .entry(*id)
            .or_insert_with(|| {
                let cell = self.storage.read_cell(*id).wait().unwrap().unwrap();
                NodeCellRef::new(Node::with_external(self.extnode_from_cell(cell)))
            })
            .clone()
    }

    fn extnode_from_cell(&self, cell: Cell) -> Box<ExtNode<KS, PS>> {
        let cell_id = cell.id();
        let cell_version = cell.header.version;
        let next = cell.data[*NEXT_PAGE_KEY_HASH].Id().unwrap();
        let prev = cell.data[*PREV_PAGE_KEY_HASH].Id().unwrap();
        let keys = &cell.data[*KEYS_KEY_HASH];
        let keys_len = keys.len().unwrap();
        let keys_array = if let Value::PrimArray(PrimitiveArray::SmallBytes(ref array)) = keys {
            array
        } else {
            panic!()
        };
        let mut key_slice = KS::init();
        let mut key_count = 0;
        for (i, key_val) in keys_array.iter().enumerate() {
            key_slice.as_slice()[i] = EntryKey::from(key_val.as_slice());
            key_count += 1;
        }
        box ExtNode {
            id: cell_id,
            keys: key_slice,
            next: self.get(next),
            prev: self.get(prev),
            len: key_count,
            dirty: false,
            right_bound: max_entry_key(), // TODO: assign a real one on reconstructing
            mark: PhantomData,
        }
    }
}
<commit_msg>make node split on heap<commit_after>use bifrost::utils::async_locks::{Mutex, RwLock, RwLockReadGuard, RwLockWriteGuard};
use bifrost::utils::fut_exec::wait;
use client::AsyncClient;
use core::borrow::BorrowMut;
use dovahkiin::types::custom_types::id::Id;
use dovahkiin::types::custom_types::map::Map;
use dovahkiin::types::type_id_of;
use dovahkiin::types::value::ToValue;
use futures::Future;
use index::btree::*;
use index::EntryKey;
use index::Slice;
use itertools::Itertools;
use owning_ref::{OwningHandle, OwningRef, RcRef};
use ram::cell::Cell;
use ram::schema::{Field, Schema};
use ram::types::*;
use std::cell::Ref;
use std::cell::RefCell;
use std::cell::RefMut;
use std::cell::UnsafeCell;
use std::collections::BTreeSet;
use std::collections::HashMap;
use std::marker::PhantomData;
use std::mem;
use std::ops::Deref;
use std::ops::DerefMut;
use std::rc::Rc;
use std::sync::atomic::AtomicBool;
use std::sync::Arc;
use utils::lru_cache::LRUCache;

const PAGE_SCHEMA: &'static str = "NEB_BTREE_PAGE";
const KEYS_FIELD: &'static str = "keys";
const NEXT_FIELD: &'static str = "next";
const PREV_FIELD: &'static str = "prev";

lazy_static! {
    static ref KEYS_KEY_HASH: u64 = key_hash(KEYS_FIELD);
    static ref NEXT_PAGE_KEY_HASH: u64 = key_hash(NEXT_FIELD);
    static ref PREV_PAGE_KEY_HASH: u64 = key_hash(PREV_FIELD);
    static ref PAGE_SCHEMA_ID: u32 = key_hash(PAGE_SCHEMA) as u32;
}

pub struct ExtNode<KS, PS>
where
    KS: Slice<EntryKey> + Debug + 'static,
    PS: Slice<NodeCellRef> + 'static,
{
    pub id: Id,
    pub keys: KS,
    pub next: NodeCellRef,
    pub prev: NodeCellRef,
    pub len: usize,
    pub dirty: bool,
    pub right_bound: EntryKey,
    mark: PhantomData<PS>,
}

pub struct ExtNodeSplit<KS, PS>
where
    KS: Slice<EntryKey> + Debug + 'static,
    PS: Slice<NodeCellRef> + 'static,
{
    pub node_2: Box<ExtNode<KS, PS>>,
    pub keys_1_len: usize,
}

impl<KS, PS> ExtNode<KS, PS>
where
    KS: Slice<EntryKey> + Debug + 'static,
    PS: Slice<NodeCellRef> + 'static,
{
    pub fn new(id: Id, right_bound: EntryKey) -> Box<Self> {
        box ExtNode {
            id,
            keys: KS::init(),
            next: Node::<KS, PS>::none_ref(),
            prev: Node::<KS, PS>::none_ref(),
            len: 0,
            dirty: false,
            right_bound,
            mark: PhantomData,
        }
    }

    pub fn to_cell(&self) -> Cell {
        let mut value = Value::Map(Map::new());
        let prev_id = read_node(&self.prev, |node: &NodeReadHandler<KS, PS>| {
            node.extnode().id
        });
        let next_id = read_node(&self.next, |node: &NodeReadHandler<KS, PS>| {
            node.extnode().id
        });

        value[*NEXT_PAGE_KEY_HASH] = Value::Id(prev_id);
        value[*PREV_PAGE_KEY_HASH] = Value::Id(next_id);
        value[*KEYS_KEY_HASH] = self.keys.as_slice_immute()[..self.len]
            .iter()
            .map(|key| SmallBytes::from_vec(key.as_slice().to_vec()))
            .collect_vec()
            .value();
        Cell::new_with_id(*PAGE_SCHEMA_ID, &self.id, value)
    }
    pub fn search(&self, key: &EntryKey) -> usize {
        self.keys.as_slice_immute()[..self.len]
            .binary_search(key)
            .unwrap_or_else(|i| i)
    }
    pub fn remove_at(&mut self, pos: usize) {
        let mut cached_len = &mut self.len;
        debug!(
            "Removing from external pos {}, len {}, key {:?}",
            pos,
            cached_len,
            self.keys.as_slice()[pos]
        );
        self.keys.remove_at(pos, cached_len);
    }

    pub fn split_insert(
        &mut self,
        key: EntryKey,
        pos: usize,
        self_ref: &NodeCellRef,
        self_next: &mut NodeWriteGuard<KS, PS>,
        tree: &BPlusTree<KS, PS>,
    ) -> (NodeCellRef, EntryKey) {
        // cached.dump();
        let pivot = self.len / 2;
        let new_page_id = tree.new_page_id();
        let mut keys_1 = &mut self.keys;
        let mut keys_2 = keys_1.split_at_pivot(pivot, self.len);
        let mut keys_1_len = pivot;
        let mut keys_2_len = self.len - pivot;
        // modify next node point previous to new node
        insert_into_split(
            key,
            keys_1,
            &mut keys_2,
            &mut keys_1_len,
            &mut keys_2_len,
            pos,
        );
        let pivot_key = keys_2.as_slice()[0].clone();
        let extnode_2: Box<ExtNode<KS, PS>> = box ExtNode {
            id: new_page_id,
            keys: keys_2,
            next: self.next.clone(),
            prev: self_ref.clone(),
            len: keys_2_len,
            dirty: true,
            right_bound: mem::replace(&mut self.right_bound, pivot_key.clone()),
            mark: PhantomData,
        };
        debug_assert!(pivot_key > smallvec!(0));
        debug_assert!(
            &pivot_key > &keys_1.as_slice()[keys_1_len - 1],
            "{:?} / {:?} @ {}",
            pivot_key,
            &keys_1.as_slice()[keys_1_len - 1],
            pos
        );
        debug_assert!(read_unchecked::<KS, PS>(&extnode_2.prev).is_ext());
        self.len = keys_1_len;
        debug!(
            "Split to left len {}, right len {}, right prev id: {:?}",
            self.len,
            extnode_2.len,
            read_unchecked::<KS, PS>(&extnode_2.prev).ext_id()
        );
        let node_2 = NodeCellRef::new(Node::with_external(extnode_2));
        {
            if !self_next.is_none() {
                self_next.extnode_mut().prev = node_2.clone();
            }
        }
        self.next = node_2.clone();

        (node_2, pivot_key)
    }

    pub fn insert(
        &mut self,
        key: &EntryKey,
        tree: &BPlusTree<KS, PS>,
        self_ref: &NodeCellRef,
        parent: &NodeCellRef,
    ) -> Option<NodeSplit<KS, PS>> {
        let key = key.clone();
        let pos = self.search(&key);
        debug_assert!(self.len <= KS::slice_len());
        debug_assert!(pos <= self.len);
        if self.len == KS::slice_len() {
            // need to split
            debug!("insert to external with split, key {:?}, pos {}", key, pos);
            let mut self_next: NodeWriteGuard<KS, PS> = write_node(&self.next);
            let mut parent_latch: NodeWriteGuard<KS, PS> = write_node(parent);
            let (node_2, pivot_key) = self.split_insert(key, pos, self_ref, &mut self_next, tree);
            Some(NodeSplit {
                new_right_node: node_2,
                pivot: pivot_key,
                parent_latch,
                left_node_latch: NodeWriteGuard::default(),
            })
        } else {
            debug!("insert to external without split at {}, key {:?}", pos, key);
            self.keys.insert_at(key, pos, &mut self.len);
            None
        }
    }

    pub fn merge_with(&mut self, right: &mut Self) {
        debug!(
            "Merge external node, left len {}:{:?}, right len {}:{:?}",
            self.len, self.keys, right.len, right.keys
        );
        let self_len = self.len;
        let new_len = self.len + right.len;
        debug_assert!(new_len <= KS::slice_len());
        for i in self.len..new_len {
            mem::swap(
                &mut self.keys.as_slice()[i],
                &mut right.keys.as_slice()[i - self_len],
            );
        }
        self.len = new_len;
    }

    pub fn remove_contains(&mut self, set: &mut BTreeSet<EntryKey>) {
        let remaining_keys = {
            let remaining = self.keys.as_slice()[..self.len]
                .iter()
                .filter(|&k| !set.remove(k))
                .collect_vec();
            if remaining.len() == self.len {
                return;
            }
            remaining.into_iter().cloned().collect_vec()
        };
        let mut self_key_slice = self.keys.as_slice();
        self.len = remaining_keys.len();
        for (i, k_ref) in remaining_keys.into_iter().enumerate() {
            self_key_slice[i] = k_ref;
        }
    }

    pub fn merge_sort(&mut self, right: &[&EntryKey]) {
        debug!("Merge sort have right nodes {:?}", right);
        let self_len_before_merge = self.len;
        debug_assert!(self_len_before_merge + right.len() <= KS::slice_len());
        let mut pos = 0;
        let mut left_pos = 0;
        let mut right_pos = 0;
        let mut left_keys = mem::replace(&mut self.keys, KS::init());
        let mut left = left_keys.as_slice();
        while left_pos < self.len && right_pos < right.len() {
            let left_key = &left[left_pos];
            let right_key = right[right_pos];
            if left_key < right_key {
                self.keys.as_slice()[pos] = left_key.clone();
                left_pos += 1;
            } else {
                self.keys.as_slice()[pos] = right_key.clone();
                right_pos += 1;
            }
            pos += 1;
        }
        for key in &left[left_pos..self.len] {
            self.keys.as_slice()[pos] = key.clone();
            pos += 1;
            left_pos += 1;
        }
        for key in &right[right_pos..] {
            self.keys.as_slice()[pos] = (*key).clone();
            pos += 1;
            right_pos += 1;
        }
        debug!(
            "Merge sorted have keys {:?}",
            &self.keys.as_slice_immute()[..pos]
        );
        self.len = pos;
        self.dirty = true;
        debug_assert_eq!(self_len_before_merge, left_pos);
        debug_assert_eq!(right.len(), right_pos);
        debug_assert_eq!(self.len, self_len_before_merge + right.len());
        debug!(
            "Merge sorted page have keys: {:?}",
            &self.keys.as_slice_immute()[..self.len]
        )
    }

    pub fn dump(&self) {
        debug!("Dumping {:?}, keys {}", self.id, self.len);
        for i in 0..KS::slice_len() {
            debug!("{}\t- {:?}", i, self.keys.as_slice_immute()[i]);
        }
    }
    pub fn is_dirty(&self) -> bool {
        unimplemented!()
    }
}

pub fn page_schema() -> Schema {
    Schema {
        id: *PAGE_SCHEMA_ID,
        name: String::from(PAGE_SCHEMA),
        key_field: None,
        str_key_field: None,
        is_dynamic: false,
        fields: Field::new(
            "*",
            0,
            false,
            false,
            Some(vec![
                Field::new(NEXT_FIELD, type_id_of(Type::Id), false, false, None),
                Field::new(PREV_FIELD, type_id_of(Type::Id), false, false, None),
                Field::new(KEYS_FIELD, type_id_of(Type::SmallBytes), false, true, None),
            ]),
        ),
    }
}

pub struct NodeCache<KS, PS>
where
    KS: Slice<EntryKey> + Debug + 'static,
    PS: Slice<NodeCellRef> + 'static,
{
    nodes: RefCell<HashMap<Id, NodeCellRef>>,
    storage: Arc<AsyncClient>,
    marker: PhantomData<(KS, PS)>,
}

impl<KS, PS> NodeCache<KS, PS>
where
    KS: Slice<EntryKey> + Debug + 'static,
    PS: Slice<NodeCellRef> + 'static,
{
    pub fn new(neb_client: &Arc<AsyncClient>) -> Self {
        NodeCache {
            nodes: RefCell::new(HashMap::new()),
            storage: neb_client.clone(),
            marker: PhantomData,
        }
    }

    pub fn get(&self, id: &Id) -> NodeCellRef {
        if id.is_unit_id() {
            return NodeCellRef::new(Node::<KS, PS>::with_none());
        }
        let mut nodes = self.nodes.borrow_mut();
        nodes
            .entry(*id)
            .or_insert_with(|| {
                let cell = self.storage.read_cell(*id).wait().unwrap().unwrap();
                NodeCellRef::new(Node::with_external(self.extnode_from_cell(cell)))
            })
            .clone()
    }

    fn extnode_from_cell(&self, cell: Cell) -> Box<ExtNode<KS, PS>> {
        let cell_id = cell.id();
        let cell_version = cell.header.version;
        let next = cell.data[*NEXT_PAGE_KEY_HASH].Id().unwrap();
        let prev = cell.data[*PREV_PAGE_KEY_HASH].Id().unwrap();
        let keys = &cell.data[*KEYS_KEY_HASH];
        let keys_len = keys.len().unwrap();
        let keys_array = if let Value::PrimArray(PrimitiveArray::SmallBytes(ref array)) = keys {
            array
        } else {
            panic!()
        };
        let mut key_slice = KS::init();
        let mut key_count = 0;
        for (i, key_val) in keys_array.iter().enumerate() {
            key_slice.as_slice()[i] = EntryKey::from(key_val.as_slice());
            key_count += 1;
        }
        box ExtNode {
            id: cell_id,
            keys: key_slice,
            next: self.get(next),
            prev: self.get(prev),
            len: key_count,
            dirty: false,
            right_bound: max_entry_key(), // TODO: assign a real one on reconstructing
            mark: PhantomData,
        }
    }
}
<|endoftext|>


<commit_before>use std::path::PathBuf;
use std::process::Command;
use std::env;
use std::io::stderr;
use std::io::Write;

pub use clap::App;

use clap::{Arg, ArgMatches};
use log;
use log::LogLevelFilter;

use configuration::Configuration;
use error::RuntimeError;
use error::RuntimeErrorKind;
use error::MapErrInto;
use logger::ImagLogger;

use libimagstore::store::Store;

#[derive(Debug)]
pub struct Runtime<'a> {
    rtp: PathBuf,
    configuration: Option<Configuration>,
    cli_matches: ArgMatches<'a>,
    store: Store,
}

impl<'a> Runtime<'a> {

    /**
     * Gets the CLI spec for the program and retreives the config file path (or uses the default on
     * in $HOME/.imag/config, $XDG_CONFIG_DIR/imag/config or from env("$IMAG_CONFIG")
     * and builds the Runtime object with it.
     *
     * The cli_spec object should be initially build with the ::get_default_cli_builder() function.
     *
     */
    pub fn new(cli_spec: App<'a, 'a>) -> Result<Runtime<'a>, RuntimeError> {
        use std::env;

        use libimagstore::hook::position::HookPosition as HP;
        use libimagstore::hook::Hook;
        use libimagstore::error::StoreErrorKind;
        use libimagstorestdhook::debug::DebugHook;
        use libimagstorestdhook::vcs::git::delete::DeleteHook as GitDeleteHook;
        use libimagstorestdhook::vcs::git::update::UpdateHook as GitUpdateHook;
        use libimagerror::trace::trace_error;
        use libimagerror::trace::trace_error_dbg;
        use libimagerror::into::IntoError;

        use configuration::error::ConfigErrorKind;

        let matches = cli_spec.get_matches();

        let is_debugging = matches.is_present("debugging");
        let is_verbose   = matches.is_present("verbosity");
        let colored      = !matches.is_present("no-color-output");

        Runtime::init_logger(is_debugging, is_verbose, colored);

        let rtp : PathBuf = matches.value_of("runtimepath")
            .map_or_else(|| {
                env::var("HOME")
                    .map(PathBuf::from)
                    .map(|mut p| { p.push(".imag"); p})
                    .unwrap_or_else(|_| {
                        panic!("You seem to be $HOME-less. Please get a $HOME before using this software. We are sorry for you and hope you have some accommodation anyways.");
                    })
            }, PathBuf::from);
        let storepath = matches.value_of("storepath")
                                .map_or_else(|| {
                                    let mut spath = rtp.clone();
                                    spath.push("store");
                                    spath
                                }, PathBuf::from);

        let configpath = matches.value_of("config")
                                .map_or_else(|| rtp.clone(), PathBuf::from);

        let cfg = match Configuration::new(&configpath) {
            Err(e) => if e.err_type() != ConfigErrorKind::NoConfigFileFound {
                return Err(RuntimeErrorKind::Instantiate.into_error_with_cause(Box::new(e)));
            } else {
                warn!("No config file found.");
                warn!("Continuing without configuration file");
                None
            },

            Ok(mut cfg) => {
                if let Err(e) = cfg.override_config(get_override_specs(&matches)) {
                    error!("Could not apply config overrides");
                    trace_error(&e);

                    // TODO: continue question (interactive)
                }

                Some(cfg)
            }
        };

        let store_config = match cfg {
            Some(ref c) => c.store_config().cloned(),
            None        => None,
        };

        if is_debugging {
            write!(stderr(), "Config: {:?}\n", cfg).ok();
            write!(stderr(), "Store-config: {:?}\n", store_config).ok();
        }

        Store::new(storepath.clone(), store_config).map(|mut store| {
            // If we are debugging, generate hooks for all positions
            if is_debugging {
                let hooks : Vec<(Box<Hook>, &str, HP)> = vec![
                    (Box::new(DebugHook::new(HP::PreCreate))          , "debug", HP::PreCreate),
                    (Box::new(DebugHook::new(HP::PostCreate))         , "debug", HP::PostCreate),
                    (Box::new(DebugHook::new(HP::PreRetrieve))        , "debug", HP::PreRetrieve),
                    (Box::new(DebugHook::new(HP::PostRetrieve))       , "debug", HP::PostRetrieve),
                    (Box::new(DebugHook::new(HP::PreUpdate))          , "debug", HP::PreUpdate),
                    (Box::new(DebugHook::new(HP::PostUpdate))         , "debug", HP::PostUpdate),
                    (Box::new(DebugHook::new(HP::PreDelete))          , "debug", HP::PreDelete),
                    (Box::new(DebugHook::new(HP::PostDelete))         , "debug", HP::PostDelete),
                ];

                // If hook registration fails, trace the error and warn, but continue.
                for (hook, aspectname, position) in hooks {
                    if let Err(e) = store.register_hook(position, &String::from(aspectname), hook) {
                        if e.err_type() == StoreErrorKind::HookRegisterError {
                            trace_error_dbg(&e);
                            warn!("Registering debug hook with store failed");
                        } else {
                            trace_error(&e);
                        };
                    }
                }
            }

            let sp = storepath;

            let hooks : Vec<(Box<Hook>, &str, HP)> = vec![
                (Box::new(GitDeleteHook::new(sp.clone(), HP::PostDelete))     , "vcs", HP::PostDelete),
                (Box::new(GitUpdateHook::new(sp, HP::PostUpdate))    , "vcs", HP::PostUpdate),
            ];

            for (hook, aspectname, position) in hooks {
                if let Err(e) = store.register_hook(position, &String::from(aspectname), hook) {
                    if e.err_type() == StoreErrorKind::HookRegisterError {
                        trace_error_dbg(&e);
                        warn!("Registering debug hook with store failed");
                    } else {
                        trace_error(&e);
                    };
                }
            }

            Runtime {
                cli_matches: matches,
                configuration: cfg,
                rtp: rtp,
                store: store,
            }
        })
        .map_err_into(RuntimeErrorKind::Instantiate)
    }

    /**
     * Get a commandline-interface builder object from `clap`
     *
     * This commandline interface builder object already contains some predefined interface flags:
     *   * -v | --verbose for verbosity
     *   * --debug for debugging
     *   * -c <file> | --config <file> for alternative configuration file
     *   * -r <path> | --rtp <path> for alternative runtimepath
     *   * --store <path> for alternative store path
     * Each has the appropriate help text included.
     *
     * The `appname` shall be "imag-<command>".
     */
    pub fn get_default_cli_builder(appname: &'a str,
                                   version: &'a str,
                                   about: &'a str)
        -> App<'a, 'a>
    {
        App::new(appname)
            .version(version)
            .author("Matthias Beyer <mail@beyermatthias.de>")
            .about(about)
            .arg(Arg::with_name(Runtime::arg_verbosity_name())
                .short("v")
                .long("verbose")
                .help("Enables verbosity")
                .required(false)
                .takes_value(false))

            .arg(Arg::with_name(Runtime::arg_debugging_name())
                .long("debug")
                .help("Enables debugging output")
                .required(false)
                .takes_value(false))

            .arg(Arg::with_name(Runtime::arg_no_color_output_name())
                .long("no-color")
                .help("Disable color output")
                .required(false)
                .takes_value(false))

            .arg(Arg::with_name(Runtime::arg_config_name())
                .long("config")
                .help("Path to alternative config file")
                .required(false)
                .takes_value(true))

            .arg(Arg::with_name(Runtime::arg_config_override_name())
                 .long("override-config")
                 .help("Override a configuration settings. Use 'key=value' pairs, where the key is a path in the TOML configuration. The value must be present in the configuration and be convertible to the type of the configuration setting. If the argument does not contain a '=', it gets ignored. Setting Arrays and Tables is not yet supported.")
                 .required(false)
                 .takes_value(true))

            .arg(Arg::with_name(Runtime::arg_runtimepath_name())
                .long("rtp")
                .help("Alternative runtimepath")
                .required(false)
                .takes_value(true))

            .arg(Arg::with_name(Runtime::arg_storepath_name())
                .long("store")
                .help("Alternative storepath. Must be specified as full path, can be outside of the RTP")
                .required(false)
                .takes_value(true))

            .arg(Arg::with_name(Runtime::arg_editor_name())
                .long("editor")
                .help("Set editor")
                .required(false)
                .takes_value(true))
    }

    pub fn arg_names() -> Vec<&'static str> {
        vec![
            Runtime::arg_verbosity_name(),
            Runtime::arg_debugging_name(),
            Runtime::arg_no_color_output_name(),
            Runtime::arg_config_name(),
            Runtime::arg_config_override_name(),
            Runtime::arg_runtimepath_name(),
            Runtime::arg_storepath_name(),
            Runtime::arg_editor_name(),
        ]
    }

    pub fn arg_verbosity_name() -> &'static str {
        "verbosity"
    }

    pub fn arg_debugging_name() -> &'static str {
        "debugging"
    }

    pub fn arg_no_color_output_name() -> &'static str {
        "no-color-output"
    }

    pub fn arg_config_name() -> &'static str {
        "config"
    }

    pub fn arg_config_override_name() -> &'static str {
        "config-override"
    }

    pub fn arg_runtimepath_name() -> &'static str {
        "runtimepath"
    }

    pub fn arg_storepath_name() -> &'static str {
        "storepath"
    }

    pub fn arg_editor_name() -> &'static str {
        "editor"
    }

    /**
     * Initialize the internal logger
     */
    fn init_logger(is_debugging: bool, is_verbose: bool, colored: bool) {
        use std::env::var as env_var;
        use env_logger;

        if env_var("IMAG_LOG_ENV").is_ok() {
            env_logger::init().unwrap();
        } else {
            let lvl = if is_debugging {
                LogLevelFilter::Debug
            } else if is_verbose {
                LogLevelFilter::Info
            } else {
                LogLevelFilter::Warn
            };

            log::set_logger(|max_log_lvl| {
                max_log_lvl.set(lvl);
                debug!("Init logger with {}", lvl);
                Box::new(ImagLogger::new(lvl.to_log_level().unwrap()).with_color(colored))
            })
            .map_err(|_| {
                panic!("Could not setup logger");
            })
            .ok();
        }
    }

    /**
     * Get the verbosity flag value
     */
    pub fn is_verbose(&self) -> bool {
        self.cli_matches.is_present("verbosity")
    }

    /**
     * Get the debugging flag value
     */
    pub fn is_debugging(&self) -> bool {
        self.cli_matches.is_present("debugging")
    }

    /**
     * Get the runtimepath
     */
    pub fn rtp(&self) -> &PathBuf {
        &self.rtp
    }

    /**
     * Get the commandline interface matches
     */
    pub fn cli(&self) -> &ArgMatches {
        &self.cli_matches
    }

    /**
     * Get the configuration object
     */
    pub fn config(&self) -> Option<&Configuration> {
        self.configuration.as_ref()
    }

    /**
     * Get the store object
     */
    pub fn store(&self) -> &Store {
        &self.store
    }

    pub fn editor(&self) -> Option<Command> {
        self.cli()
            .value_of("editor")
            .map(String::from)
            .or({
                match self.configuration {
                    Some(ref c) => c.editor().cloned(),
                    _ => None,
                }
            })
            .or(env::var("EDITOR").ok())
            .map(Command::new)
    }
}

fn get_override_specs(matches: &ArgMatches) -> Vec<String> {
    matches
        .values_of("config-override")
        .map(|values| {
             values
             .filter(|s| {
                 let b = s.contains("=");
                 if !b { warn!("override '{}' does not contain '=' - will be ignored!", s); }
                 b
             })
             .map(String::from)
             .collect()
        })
        .unwrap_or(vec![])
}

<commit_msg>Fix git hook registration fail warn message.<commit_after>use std::path::PathBuf;
use std::process::Command;
use std::env;
use std::io::stderr;
use std::io::Write;

pub use clap::App;

use clap::{Arg, ArgMatches};
use log;
use log::LogLevelFilter;

use configuration::Configuration;
use error::RuntimeError;
use error::RuntimeErrorKind;
use error::MapErrInto;
use logger::ImagLogger;

use libimagstore::store::Store;

#[derive(Debug)]
pub struct Runtime<'a> {
    rtp: PathBuf,
    configuration: Option<Configuration>,
    cli_matches: ArgMatches<'a>,
    store: Store,
}

impl<'a> Runtime<'a> {

    /**
     * Gets the CLI spec for the program and retreives the config file path (or uses the default on
     * in $HOME/.imag/config, $XDG_CONFIG_DIR/imag/config or from env("$IMAG_CONFIG")
     * and builds the Runtime object with it.
     *
     * The cli_spec object should be initially build with the ::get_default_cli_builder() function.
     *
     */
    pub fn new(cli_spec: App<'a, 'a>) -> Result<Runtime<'a>, RuntimeError> {
        use std::env;

        use libimagstore::hook::position::HookPosition as HP;
        use libimagstore::hook::Hook;
        use libimagstore::error::StoreErrorKind;
        use libimagstorestdhook::debug::DebugHook;
        use libimagstorestdhook::vcs::git::delete::DeleteHook as GitDeleteHook;
        use libimagstorestdhook::vcs::git::update::UpdateHook as GitUpdateHook;
        use libimagerror::trace::trace_error;
        use libimagerror::trace::trace_error_dbg;
        use libimagerror::into::IntoError;

        use configuration::error::ConfigErrorKind;

        let matches = cli_spec.get_matches();

        let is_debugging = matches.is_present("debugging");
        let is_verbose   = matches.is_present("verbosity");
        let colored      = !matches.is_present("no-color-output");

        Runtime::init_logger(is_debugging, is_verbose, colored);

        let rtp : PathBuf = matches.value_of("runtimepath")
            .map_or_else(|| {
                env::var("HOME")
                    .map(PathBuf::from)
                    .map(|mut p| { p.push(".imag"); p})
                    .unwrap_or_else(|_| {
                        panic!("You seem to be $HOME-less. Please get a $HOME before using this software. We are sorry for you and hope you have some accommodation anyways.");
                    })
            }, PathBuf::from);
        let storepath = matches.value_of("storepath")
                                .map_or_else(|| {
                                    let mut spath = rtp.clone();
                                    spath.push("store");
                                    spath
                                }, PathBuf::from);

        let configpath = matches.value_of("config")
                                .map_or_else(|| rtp.clone(), PathBuf::from);

        let cfg = match Configuration::new(&configpath) {
            Err(e) => if e.err_type() != ConfigErrorKind::NoConfigFileFound {
                return Err(RuntimeErrorKind::Instantiate.into_error_with_cause(Box::new(e)));
            } else {
                warn!("No config file found.");
                warn!("Continuing without configuration file");
                None
            },

            Ok(mut cfg) => {
                if let Err(e) = cfg.override_config(get_override_specs(&matches)) {
                    error!("Could not apply config overrides");
                    trace_error(&e);

                    // TODO: continue question (interactive)
                }

                Some(cfg)
            }
        };

        let store_config = match cfg {
            Some(ref c) => c.store_config().cloned(),
            None        => None,
        };

        if is_debugging {
            write!(stderr(), "Config: {:?}\n", cfg).ok();
            write!(stderr(), "Store-config: {:?}\n", store_config).ok();
        }

        Store::new(storepath.clone(), store_config).map(|mut store| {
            // If we are debugging, generate hooks for all positions
            if is_debugging {
                let hooks : Vec<(Box<Hook>, &str, HP)> = vec![
                    (Box::new(DebugHook::new(HP::PreCreate))          , "debug", HP::PreCreate),
                    (Box::new(DebugHook::new(HP::PostCreate))         , "debug", HP::PostCreate),
                    (Box::new(DebugHook::new(HP::PreRetrieve))        , "debug", HP::PreRetrieve),
                    (Box::new(DebugHook::new(HP::PostRetrieve))       , "debug", HP::PostRetrieve),
                    (Box::new(DebugHook::new(HP::PreUpdate))          , "debug", HP::PreUpdate),
                    (Box::new(DebugHook::new(HP::PostUpdate))         , "debug", HP::PostUpdate),
                    (Box::new(DebugHook::new(HP::PreDelete))          , "debug", HP::PreDelete),
                    (Box::new(DebugHook::new(HP::PostDelete))         , "debug", HP::PostDelete),
                ];

                // If hook registration fails, trace the error and warn, but continue.
                for (hook, aspectname, position) in hooks {
                    if let Err(e) = store.register_hook(position, &String::from(aspectname), hook) {
                        if e.err_type() == StoreErrorKind::HookRegisterError {
                            trace_error_dbg(&e);
                            warn!("Registering debug hook with store failed");
                        } else {
                            trace_error(&e);
                        };
                    }
                }
            }

            let sp = storepath;

            let hooks : Vec<(Box<Hook>, &str, HP)> = vec![
                (Box::new(GitDeleteHook::new(sp.clone(), HP::PostDelete))     , "vcs", HP::PostDelete),
                (Box::new(GitUpdateHook::new(sp, HP::PostUpdate))    , "vcs", HP::PostUpdate),
            ];

            for (hook, aspectname, position) in hooks {
                if let Err(e) = store.register_hook(position, &String::from(aspectname), hook) {
                    if e.err_type() == StoreErrorKind::HookRegisterError {
                        trace_error_dbg(&e);
                        warn!("Registering git hook with store failed");
                    } else {
                        trace_error(&e);
                    };
                }
            }

            Runtime {
                cli_matches: matches,
                configuration: cfg,
                rtp: rtp,
                store: store,
            }
        })
        .map_err_into(RuntimeErrorKind::Instantiate)
    }

    /**
     * Get a commandline-interface builder object from `clap`
     *
     * This commandline interface builder object already contains some predefined interface flags:
     *   * -v | --verbose for verbosity
     *   * --debug for debugging
     *   * -c <file> | --config <file> for alternative configuration file
     *   * -r <path> | --rtp <path> for alternative runtimepath
     *   * --store <path> for alternative store path
     * Each has the appropriate help text included.
     *
     * The `appname` shall be "imag-<command>".
     */
    pub fn get_default_cli_builder(appname: &'a str,
                                   version: &'a str,
                                   about: &'a str)
        -> App<'a, 'a>
    {
        App::new(appname)
            .version(version)
            .author("Matthias Beyer <mail@beyermatthias.de>")
            .about(about)
            .arg(Arg::with_name(Runtime::arg_verbosity_name())
                .short("v")
                .long("verbose")
                .help("Enables verbosity")
                .required(false)
                .takes_value(false))

            .arg(Arg::with_name(Runtime::arg_debugging_name())
                .long("debug")
                .help("Enables debugging output")
                .required(false)
                .takes_value(false))

            .arg(Arg::with_name(Runtime::arg_no_color_output_name())
                .long("no-color")
                .help("Disable color output")
                .required(false)
                .takes_value(false))

            .arg(Arg::with_name(Runtime::arg_config_name())
                .long("config")
                .help("Path to alternative config file")
                .required(false)
                .takes_value(true))

            .arg(Arg::with_name(Runtime::arg_config_override_name())
                 .long("override-config")
                 .help("Override a configuration settings. Use 'key=value' pairs, where the key is a path in the TOML configuration. The value must be present in the configuration and be convertible to the type of the configuration setting. If the argument does not contain a '=', it gets ignored. Setting Arrays and Tables is not yet supported.")
                 .required(false)
                 .takes_value(true))

            .arg(Arg::with_name(Runtime::arg_runtimepath_name())
                .long("rtp")
                .help("Alternative runtimepath")
                .required(false)
                .takes_value(true))

            .arg(Arg::with_name(Runtime::arg_storepath_name())
                .long("store")
                .help("Alternative storepath. Must be specified as full path, can be outside of the RTP")
                .required(false)
                .takes_value(true))

            .arg(Arg::with_name(Runtime::arg_editor_name())
                .long("editor")
                .help("Set editor")
                .required(false)
                .takes_value(true))
    }

    pub fn arg_names() -> Vec<&'static str> {
        vec![
            Runtime::arg_verbosity_name(),
            Runtime::arg_debugging_name(),
            Runtime::arg_no_color_output_name(),
            Runtime::arg_config_name(),
            Runtime::arg_config_override_name(),
            Runtime::arg_runtimepath_name(),
            Runtime::arg_storepath_name(),
            Runtime::arg_editor_name(),
        ]
    }

    pub fn arg_verbosity_name() -> &'static str {
        "verbosity"
    }

    pub fn arg_debugging_name() -> &'static str {
        "debugging"
    }

    pub fn arg_no_color_output_name() -> &'static str {
        "no-color-output"
    }

    pub fn arg_config_name() -> &'static str {
        "config"
    }

    pub fn arg_config_override_name() -> &'static str {
        "config-override"
    }

    pub fn arg_runtimepath_name() -> &'static str {
        "runtimepath"
    }

    pub fn arg_storepath_name() -> &'static str {
        "storepath"
    }

    pub fn arg_editor_name() -> &'static str {
        "editor"
    }

    /**
     * Initialize the internal logger
     */
    fn init_logger(is_debugging: bool, is_verbose: bool, colored: bool) {
        use std::env::var as env_var;
        use env_logger;

        if env_var("IMAG_LOG_ENV").is_ok() {
            env_logger::init().unwrap();
        } else {
            let lvl = if is_debugging {
                LogLevelFilter::Debug
            } else if is_verbose {
                LogLevelFilter::Info
            } else {
                LogLevelFilter::Warn
            };

            log::set_logger(|max_log_lvl| {
                max_log_lvl.set(lvl);
                debug!("Init logger with {}", lvl);
                Box::new(ImagLogger::new(lvl.to_log_level().unwrap()).with_color(colored))
            })
            .map_err(|_| {
                panic!("Could not setup logger");
            })
            .ok();
        }
    }

    /**
     * Get the verbosity flag value
     */
    pub fn is_verbose(&self) -> bool {
        self.cli_matches.is_present("verbosity")
    }

    /**
     * Get the debugging flag value
     */
    pub fn is_debugging(&self) -> bool {
        self.cli_matches.is_present("debugging")
    }

    /**
     * Get the runtimepath
     */
    pub fn rtp(&self) -> &PathBuf {
        &self.rtp
    }

    /**
     * Get the commandline interface matches
     */
    pub fn cli(&self) -> &ArgMatches {
        &self.cli_matches
    }

    /**
     * Get the configuration object
     */
    pub fn config(&self) -> Option<&Configuration> {
        self.configuration.as_ref()
    }

    /**
     * Get the store object
     */
    pub fn store(&self) -> &Store {
        &self.store
    }

    pub fn editor(&self) -> Option<Command> {
        self.cli()
            .value_of("editor")
            .map(String::from)
            .or({
                match self.configuration {
                    Some(ref c) => c.editor().cloned(),
                    _ => None,
                }
            })
            .or(env::var("EDITOR").ok())
            .map(Command::new)
    }
}

fn get_override_specs(matches: &ArgMatches) -> Vec<String> {
    matches
        .values_of("config-override")
        .map(|values| {
             values
             .filter(|s| {
                 let b = s.contains("=");
                 if !b { warn!("override '{}' does not contain '=' - will be ignored!", s); }
                 b
             })
             .map(String::from)
             .collect()
        })
        .unwrap_or(vec![])
}

<|endoftext|>