All ``feature_ids`` must be present as tip names in ``phylogeny``. Tips missing when converting the newick tree to qza format

Hi qimme team,

I'm tying to calculate food diversity in diet intake data using a tree I build with the US FNDDS food database. I'm using qimme2-2019.10 conda environment.

My command of calculating alpha diversity with phylogenetics:
qiime diversity alpha-phylogenetic
--i-table all_patients_food_counts_table_w_maindescrip_featurename.qza
--i-phylogeny ../source/output_food_tree_datatree.qza
--p-metric faith_pd
--o-alpha-diversity faith_pd_vector.qza

I have the newick tree built using the R data.tree package and then convert to qza format using commad:
qiime tools import
--input-path output_food_tree_datatree.newick
--output-path output_food_tree_datatree.qza
--type 'Phylogeny[Rooted]'
There are only 483 tips in the qza format, but I have 498 tips in the newick format.

Also, I have a feature table with 495 features, all of them exist in the 498, so there are 12 features missing in the qza tree format and that is causing the problem:
All feature_ids must be present as tip names in phylogeny. feature_ids not corresponding to tip names (n=483): Pudding__ready_to_eat__low_calorie__containing_artificial_sweetener__chocolate Vegetable_soup__home_recipe Beef_stew_with_potatoes_and_vegetables_including_carrots__broccoli__and_or_dark_green_leafy__tomato_based_sauce Fruit_cocktail__cooked_or_canned__in_light_syrup Water__bottled__flavored__Glaceau_Vitamin_Water_ Milk__lactose_free__fat_free__skim_ Muffin__English Graham_crackers Italian_dressing__made_with_vinegar_and_oil Potato__roasted__from_fresh__peel_eaten__made_with_oil Gravy__mushroom Brussels_sprouts__cooked__from_fresh__fat_not_added_in_cooking Turkey_sandwich__with_spread Spinach__raw Hot_chocolate___Cocoa__made_with_no_sugar_added_dry_mix_and_water Grapes__raw__NS_as_to_type Tilapia__baked_or_broiled__made_with_oil Cream__heavy Sugar_substitute__sucralose__powder Pork_chop__broiled_or_baked__lean_and_fat_eaten Cookie__animal Ripe_plantain_fritters__Puerto_Rican_style Peppers__green__cooked__made_with_oil Soft_drink__fruit_flavored__diet__caffeine_free Tortilla__flour Spinach__cooked__from_fresh__made_with_oil Milk__lactose_free__whole Pasta_with_tomato_based_sauce__home_recipe Cereal__Kelloggs_Corn_Flakes_ Chocolate__sweet_or_dark Meat_loaf_made_with_beef Ice_pop Tea__hot__leaf__green Cabbage__Chinese__cooked__fat_not_added_in_cooking Orange__raw Pumpkin__cooked__from_canned__made_with_oil Chicken_or_turkey_pot_pie Oatmeal__instant__plain__made_with_water__fat_not_added_in_cooking Cereal__Post_Shredded_Wheat_ Ham__prepackaged_or_deli__luncheon_meat Pudding__ready_to_eat__tapioca Romaine_lettuce__raw Teriyaki_sauce Turkey_or_chicken_sausage Waffle__NFS Tea__hot__leaf__black Pasta_with_tomato_based_sauce_and_meat__home_recipe Cantaloupe__raw Quesadilla_with_vegetables_and_chicken Cereal__Kelloggs_Low_Fat_Granola_with_Raisins_ Stuffed_pepper__with_rice_and_meat Cake__pound__with_icing_or_filling Nachos_with_cheese Mixed_nuts__with_peanuts__salted Tomato_soup__cream_of__prepared_with_milk Cheese__Provolone Noodles__cooked Roll__white__hamburger_bun Barley_soup__home_recipe__canned__or_ready_to_serve Muffin__fruit Chicken_sandwich__with_spread Potato__scalloped__from_fresh Turkey_or_chicken_burger__plain__on_white_bun Cheese__Gruyere Milk_shake__home_recipe__flavors_other_than_chocolate Broccoli__cooked__from_fresh__with_cheese_sauce Flounder__baked_or_broiled__made_with_oil Soybean_curd_cheese Potato__baked__peel_eaten Mixed_vegetables__cooked__NS_as_to_form__made_with_oil Fruit_juice_drink Doughnut__cake_type Yogurt__Greek__low_fat_milk__plain Cereal__Kelloggs_Rice_Krispies_ Cheese__provolone__reduced_fat Ravioli__meat_filled__no_sauce Orange__mandarin__canned_or_frozen__drained Swedish_meatballs_with_cream_or_white_sauce Cookie__gluten_free Banana__raw French_or_Catalina_dressing Chicken_breast__baked__broiled__or_roasted_with_marinade__skin_eaten__from_raw Squash__winter_type__soup__home_recipe__canned__or_ready_to_serve Chicken_fillet__breaded__fried__sandwich_with_lettuce__tomato_and_spread Hamburger__1_medium_patty__plain__on_white_bun Gravy__poultry__home_recipe Stuffed_pepper__with_rice__meatless Tortilla_chips__flavored Chicken__wings__with_other_sauces_or_seasoning__from_fast_food___restaurant Blueberries__frozen__unsweetened Beef_and_vegetables_including_carrots__broccoli__and_or_dark_green_leafy__no_potatoes__soy_based_sauce Fruit_smoothie__with_whole_fruit_and_dairy Sea_bass__baked_or_broiled__fat_added_in_cooking Rice__white__cooked__fat_added_in_cooking__made_with_oil Salsa__red__homemade Nutritional_powder_mix__Muscle_Milk_ Carrots__cooked__from_fresh__fat_not_added_in_cooking Crepe__plain Prune_juice__100_ Soft_drink__ginger_ale__diet Cereal__Post_Honeycomb_ Vegetable_noodle_soup__canned__prepared_with_water__or_ready_to_serve Soft_drink__fruit_flavored__caffeine_free Rice__cooked__NFS Beef__broth__bouillon__or_consomme Shrimp__baked_or_broiled__made_with_oil Strudel__apple Cod__baked_or_broiled__made_with_oil Corned_beef__cooked__lean_and_fat_eaten Pepperoni__NFS Rice__brown__cooked__fat_added_in_cooking__made_with_oil Pretzels__soft__ready_to_eat__salted__no_butter Tomato_and_vegetable_juice__100___low_sodium Crackers__cheese__Cheez_It_ Coconut_milk Honeydew_melon__raw Cake_or_cupcake__yellow__with_icing_or_filling Honey_mustard_dressing Meat_loaf_made_with_chicken_or_turkey Potato__mashed__from_fast_food__with_gravy Soft_drink__ginger_ale Mayonnaise__regular Paella_with_seafood Soft_drink__cola__diet Soy_sauce Matzo_ball_soup Peach__cooked_or_canned__drained_solids Yogurt_parfait__low_fat__with_fruit Cookie__brownie__without_icing Potato_soup__prepared_with_water Nutritional_drink_or_shake__ready_to_drink__sugar_free__Glucerna_ Pancakes__plain Coffee__brewed__decaffeinated Cereal__Kelloggs_Low_Fat_Granola_ Quesadilla__just_cheese__meatless Gravy__beef_or_meat Pancakes__gluten_free Tomatoes__NS_as_to_form__broiled Guacamole__NFS Spaghetti_sauce Clam_chowder__Manhattan Crackers__wheat Soy_milk__light__chocolate Cereal__raisin_bran Reeses_Peanut_Butter_Cup Cabbage_salad_or_coleslaw__made_with_creamy_dressing Nutritional_drink_or_shake__ready_to_drink__Ensure_Plus_ Chicken_breast__grilled_without_sauce__skin_not_eaten Egg_white_omelet__scrambled__or_fried__made_without_fat Beef_steak__broiled_or_baked__lean_and_fat_eaten Popcorn__microwave__plain Cookie__almond Cauliflower__cooked__from_fresh__fat_not_added_in_cooking Duck_sauce Cake_or_cupcake__white__with_icing_or_filling Meat_loaf_made_with_beef__with_tomato_based_sauce Minestrone_soup__home_recipe Chocolate_milk__ready_to_drink__fat_free Cookie__marshmallow__with_rice_cereal__no_bake French_toast__plain Grape_juice__100_ Coffee__brewed Lasagna_with_meat Cookie__chocolate_sandwich Cookie__molasses Pad_Thai_with_seafood Cake_or_cupcake__chocolate__devils_food_or_fudge__with_icing_or_filling Sugar_substitute__saccharin__powder Tortilla_chips__nacho_cheese_flavor__Doritos_ Spaghetti_sauce_with_meat Tea__hot__leaf__black__decaffeinated Tomato_catsup Cookie__fig_bar Tomato_soup__prepared_with_water__or_ready_to_serve Cereal__General_Mills_Cheerios_ Salmon__baked_or_broiled__made_with_oil Cheese__Ricotta Tomatoes__red__dried Sour_cream Potato_chips__barbecue_flavored Bread__white Olive_oil Butter__NFS Beef_sausage Cookie__vanilla_wafer Milk__whole Pasta_with_cream_sauce__home_recipe Wrap_sandwich__filled_with_meat__poultry__or_fish__vegetables__and_cheese Cheese__NFS Cheese__Mozzarella__NFS Mushroom_soup__cream_of__prepared_with_milk Apple__raw Lettuce__salad_with_assorted_vegetables_including_tomatoes_and_or_carrots__no_dressing Pie__apple__two_crust Water__bottled__unsweetened Bagel__with_raisins Squash__summer__yellow_or_green__cooked__from_fresh__made_with_oil Muffin__carrot Rice__white__cooked__fat_not_added_in_cooking Blue_or_roquefort_cheese_dressing Onions__cooked__from_fresh__made_with_oil Potato__baked__peel_not_eaten Cranberries__dried Cookie__butter_or_sugar Crackers__oyster Lentil_soup__home_recipe__canned__or_ready_to_serve Soybean_curd__breaded__fried Yellow_rice__cooked__fat_added_in_cooking Vegetable_soup__chunky_style Pretzels__hard__NFS Lobster_bisque Cheese__Cheddar Black_bean_soup__home_recipe__canned_or_ready_to_serve Broccoli_soup__prepared_with_milk__home_recipe__canned_or_ready_to_serve Rice_noodles__cooked Coconut__packaged Egg_roll__meatless Cream_of_wheat__instant__made_with_milk__fat_not_added_in_cooking Yogurt__whole_milk__fruit Clam_chowder__New_England__prepared_with_milk Cheese_spread__cream_cheese__regular Cereal__General_Mills_Cheerios_Fruity_ Peanut_butter Pickles__mixed Squash__summer__yellow__raw Potato_salad__German_style Pesto_sauce Potato__french_fries__restaurant Egg_salad__made_with_mayonnaise Steak_and_cheese_submarine_sandwich__with_fried_peppers_and_onions__on_roll Syrup__pancake__reduced_calorie Sugar_substitute__aspartame__powder Beef__roast__roasted__lean_only_eaten Sauerkraut__cooked__fat_not_added_in_cooking Cream__half_and_half Asian_chicken_or_turkey_garden_salad_with_crispy_noodles__chicken_and_or_turkey__lettuce__fruit__nuts__crispy_noodles__no_dressing Yogurt__low_fat_milk__fruit Nutritional_drink_or_shake__ready_to_drink__Ensure_ Cheese__Parmesan__dry_grated Artichoke_salad_in_oil Carbonated_water__unsweetened Bread__rye Pancakes__whole_grain Coffee_cake__crumb_or_quick_bread_type Soup__mostly_noodles Mushrooms__raw Creamy_dressing Crackers__butter__Ritz_ Milk__fat_free__skim_ Tzatziki_dip Tartar_sauce Celery__raw Manicotti__cheese_filled__with_tomato_sauce__meatless Cereal__Quaker_Capn_Crunch_ Kit_Kat Oatmeal__instant__fruit_flavored__fat_not_added_in_cooking Pasta__vegetable__cooked Chocolate_milk__ready_to_drink__whole Potato__mashed__NFS Margarine__stick__salted Onions__mature__raw Split_pea_soup Roll__white__hard Strawberries__raw Milk_chocolate_candy__plain Pudding__rice Rice_pilaf Baked_beans__NFS Ice__fruit Egg_roll__with_beef_and_or_pork Cottage_cheese__farmers Caesar_dressing Egg__whole__fried_with_oil Hummus__plain Lettuce__raw Butter__stick__salted Vegetable_broth__bouillon Beef_steak__braised__lean_and_fat_eaten Watermelon__raw Hard_candy Potato_chowder Barley__fat_added_in_cooking Coffee_creamer__liquid Cheese_dip Wrap_sandwich__filled_with_vegetables Crab_cake Apple_juice__100_ Corn__cooked__NS_as_to_form__NS_as_to_color__made_with_oil Tortilla_chips__plain Applesauce__stewed_apples__unsweetened Mushrooms__cooked__from_fresh__made_with_oil Cake_or_cupcake__carrot__with_icing_or_filling Chicken_or_turkey_and_rice_with_soy_based_sauce Roll__white__soft Vegetarian_burger_or_patty__meatless__no_bun Cereal_or_granola_bar__Kashi_Crunchy_ Gelatin_dessert__dietetic__sweetened_with_low_calorie_sweetener Pear__cooked_or_canned__drained_solids Egg_white_omelet__scrambled__or_fried__made_with_oil Sweet_potato__baked__peel_not_eaten__made_with_oil Potato_chips__plain Plantain_chips Cheese__Muenster Shrimp__coated__fried__from_fast_food___restaurant Yogurt__whole_milk__plain Cereal__General_Mills_Cheerios_Apple_Cinnamon_ Corn_with_peppers__red_or_green__cooked__made_with_oil Chicken_or_turkey_caesar_garden_salad__chicken_and_or_turkey__lettuce__tomato__cheese__no_dressing Soy_milk__chocolate Pad_Thai__meatless Beef_short_ribs__boneless__with_barbecue_sauce__potatoes__vegetable__frozen_meal Fruit_snacks_candy__with_high_vitamin_C Pudding__ready_to_eat__chocolate Popcorn__microwave__low_sodium Red_beans_and_rice Bread_stuffing Pasta__cooked Russian_dressing Beans__string__green__cooked__from_fresh__fat_not_added_in_cooking Chickpeas__canned__drained__fat_not_added_in_cooking Cookie__ladyfinger Cornbread_muffin__stick__round Sugar__white__granulated_or_lump Blueberries__raw Bread__wheat_or_cracked_wheat Granola__homemade Chicken_or_turkey_parmigiana Tomatoes__raw Shepherds_pie_with_beef Cheese_flavored_corn_snacks__Cheetos_ Cream_of_wheat__regular_or_quick__made_with_milk__NS_as_to_fat_added_in_cooking Vegetable_combination__including_carrots__broccoli__and_or_dark_green_leafy__cooked__with_soy_based_sauce Eggplant_parmesan_casserole__regular Pizza__cheese__from_frozen__thin_crust Nutritional_powder_mix__whey_based__NFS Cheese_spread__cream_cheese__light_or_lite Chocolate_milk__ready_to_drink__reduced_fat Caesar_salad__with_romaine__no_dressing Cranberry_juice_blend__100__juice Turkey__light_and_dark_meat__roasted__skin_not_eaten Couscous__plain__cooked Blueberries__cooked_or_canned__in_heavy_syrup Barbecue_sauce Lamb_chop__NS_as_to_cut__cooked__lean_and_fat_eaten Sports_drink__low_calorie__Gatorade_G2_ Light_ice_cream__chocolate Pudding__ready_to_eat__flavors_other_than_chocolate Tea__iced__bottled__black__diet Peas__green__cooked__from_fresh__fat_not_added_in_cooking Grilled_cheese_sandwich__American_cheese__on_white_bread Maple_syrup Green_peppers_and_onions__cooked__made_with_oil Lemon_butter_sauce Cheese__Feta Orange_juice__100____freshly_squeezed Pizza__no_cheese__thin_crust Peppers__red__cooked__made_with_oil Fruit_flavored_smoothie_drink__frozen__no_dairy Sugar__brown Cereal__Kelloggs_Corn_Pops_ Soy_milk Breakfast_tart Roll__white__hot_dog_bun Salsa__red__commercially_prepared Cheese__cottage__low_fat Cookie__oatmeal__with_raisins Italian_Wedding_Soup Olives__black Pork_bacon__NS_as_to_fresh__smoked_or_cured__cooked Nutrition_bar__PowerBar_ Gelatin_dessert Asparagus__cooked__from_fresh__fat_not_added_in_cooking Macaroni_or_noodles_with_cheese Tuna_salad__made_with_mayonnaise Tea__hot__chamomile Milk__reduced_fat__2__ Lo_mein__meatless Chocolate__semi_sweet_morsel Pineapple__raw Sports_drink__Gatorade_G_ Beef__shortribs__cooked__lean_and_fat_eaten Crackers__sandwich__peanut_butter_filled Frankfurter_or_hot_dog__beef Chili_con_carne_with_beans__canned Cobbler__berry Broccoli__cooked__from_fresh__fat_not_added_in_cooking Stuffed_chicken__drumstick_or_breast__Puerto_Rican_style Chicken_tenders_or_strips__breaded__from_other_sources Ravioli__cheese_filled__no_sauce Quinoa__fat_added_in_cooking Grits__regular_or_quick__made_with_water__fat_not_added_in_cooking Frankfurter_or_hot_dog__beef_and_pork Chicken_breast__baked__coated__skin___coating_eaten Cheese__Swiss Potato__baked__peel_eaten__with_cheese Cookie__chocolate_wafer Tortellini__cheese_filled__no_sauce Cereal__Kashi_Heart_to_Heart_Oat_Flakes_and_Blueberry_Clusters_ Cilantro__raw Yogurt__whole_milk__flavors_other_than_fruit Jelly__sugar_free__all_flavors Topping__butterscotch_or_caramel Cornmeal_mush__NS_as_to_fat_added_in_cooking Cheese__American Turkey__ground Pretzels__hard__plain__lightly_salted Pudding__ready_to_eat__low_calorie__containing_artificial_sweetener__flavors_other_than_chocolate Snowpea__cooked__from_fresh__made_with_butter Muffin__chocolate_chip Chicken_or_turkey_noodle_soup__home_recipe Bread__multigrain Pasta_with_tomato_based_sauce__ready_to_heat Ice_cream__no_sugar_added__flavors_other_than_chocolate Fruit_juice_drink__with_high_vitamin_C Nutritional_drink_or_shake__ready_to_drink__Boost_Plus_ Peanut_butter_and_jelly_sandwich__with_regular_peanut_butter__reduced_sugar_jelly__on_whole_wheat_bread Lemonade__fruit_juice_drink Potato__boiled__from_fresh___peel_eaten__fat_not_added_in_cooking Egg__whole__boiled_or_poached Cookie__chocolate_chip Cookie__shortbread Egg_omelet_or_scrambled_egg__made_with_oil Danish_pastry__plain_or_spice Canola_and_soybean_oil Fruit__NS_as_to_type Chicken_or_turkey_rice_soup__home_recipe Soft_drink__cola Pineapple_juice__100_ Spinach_quiche__meatless Kale__cooked__from_fresh__fat_not_added_in_cooking Dumpling__vegetable Pancakes__with_fruit Ice_cream__no_sugar_added__chocolate Fish_stick__patty_or_nugget_from_fast_food Milk_shake__home_recipe__chocolate Doughnut__chocolate__cake_type__with_chocolate_icing Yogurt__Greek__nonfat_milk__plain Pork_sausage Stuffed_shells__cheese_filled__with_tomato_sauce__meatless Bread__fruit Crackers__NFS Cheese__Mozzarella__whole_milk Jelly__all_flavors Ravioli__NS_as_to_filling__no_sauce Butterscotch_hard_candy Beans__canned__drained__NS_as_to_type__fat_not_added_in_cooking Chicken_or_turkey_broth__bouillon__or_consomme Vegetable_oil__NFS Jam__preserve__all_flavors Almond_milk__sweetened Ice_cream__regular__chocolate Vegetables__NS_as_to_type__cooked__made_with_oil Cereal__Kelloggs_Frosted_Flakes_ Roll__whole_wheat__hamburger_bun Yogurt__Greek__nonfat_milk__fruit Dill_dip__regular Prune__dried__cooked__with_sugar Oatmeal__instant__fruit_flavored__NS_as_to_fat_added_in_cooking Vodka_sauce_with_tomatoes_and_cream Cheese__Cheddar__reduced_fat Crackers__cheese__Goldfish_ Bread__French_or_Vienna TWIX_Caramel_Cookie_Bars Avocado__raw Turkey_bacon__cooked Carrots__raw Cheese__Colby White_beans__dry__cooked__NS_as_to_fat_added_in_cooking Chicken_breast__baked__broiled__or_roasted__skin_eaten__from_raw Carrot_soup__cream_of__prepared_with_milk__home_recipe__canned_or_ready_to_serve Snack_mix__plain__Chex_Mix_ Ice_cream__regular__flavors_other_than_chocolate Fruit_cocktail__cooked_or_canned__NS_as_to_sweetened_or_unsweetened__sweetened__NS_as_to_type_of_sweetener Potato__hash_brown__NFS Tofu_and_vegetables_excluding_carrots__broccoli__and_dark_green_leafy__no_potatoes__with_soy_based_sauce Cheese__Fontina Lemon_juice__100___freshly_squeezed Sorbet__fruit__noncitrus_flavor Bread__Irish_soda Tea__iced__bottled__black Bagel__whole_wheat Cauliflower__cooked__from_fresh__made_with_oil Coconut_water__unsweetened

I've attached my newick tree. Can you help me check what is causing the problem?

Thank yououtput_food_tree_datatree.newick|attachment (55.4 KB)
Angel

I’ve looked more into this issue. And I’ve found the ultimate problem

In my newick tree, the tips are like this: “Cauliflower__cooked__from_fresh__made”

but when its read into the program and the function _validate_otu_ids_and_tree extracts the tips in the format of absolutely no underscores. All of the underscores have been replaced with spaces.

That is why it doesn’t match with my features in the feature table, since I have underscores.

I think to get it to work, I can just have features having no underscores as well.

But if anybody is interested in clearing the confusion. Welcome!

Otherwise, this issue could be considered resolved.

Angel

1 Like

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.