Monte Carlo tree search
For balancing exploitation and exploration, UCT (Upper Confidence Bound applied to trees) algorithm was introduced by Levente Kocsis and Csaba Szepesvári.
Kocsis, L. and Szepesvári, C., 2006, September. Bandit based monte-carlo planning. In European conference on machine learning (pp. 282-293). Springer, Berlin, Heidelberg.
Related:
Related Pages
- →gp-ucb×ガウス過程回帰×UCB1×gaussian_process_optimization_in_the_bandit_setting:_no_regret_and_experimental_design×gaussian_process×randomized_gaussian_process_upper_confidence_bound_with_tighter_bayesian_regret_bounds→
- →広聴ai×抽出された知見×大量のオブジェクト×主観的興味深さ×今持っている知識の一歩先の知識しか受け止められない×ガウス過程回帰×利用と探索のトレードオフ×興味深さ×主観×cartographer×多様性ペナルティ×maximal_marginal_relevance×upper_confidence_bound×random_kitchen_sinks→
- →(0.3)_structure_of_this_book×(0)_preface×(0.0)_what_is_intellitech?×(0.1)_purpose_of_this_book×(0.1.1)_what_is_intellectual_production×(0.1.2)_benefits_of_reading_this_book×(0.2)_how_to_learn_programming×(0.2.1)_collect_information_concretely×(0.2.2)_compare_and_find_patterns×(0.2.3)_practice_and_verification×(0.4)_acknowledgments×(1)_how_to_learn×(1.1)_the_learning_cycle×(1.1.1)_information_gathering×(1.1.2)_modeling_and_abstraction×(1.1.3)_practice_and_verification×(1.2)_driving_force_to_cycle:_motivation×(1.2.1)_difference_between_student_learning_and_university_learning×(1.2.1.1)_teacher_gives_you_textbooks×(1.2.1.2)_how_much_time_can_you_spend_for_learning?×(1.2.1.3)_who_pays_money_to_learn?×(1.2.1.4)_counterwind×(1.2.2)_how_to_keep_motivated?×(1.2.2.1)_clarify_the_goal×(1.2.2.2)_make_the_goal_closer_-_the_effect_of_tutorial×(column)_smart_criteria×(1.2.3)_should_i_go_to_university?×(1.2.3.1)_easy_ways×(1.2.4)_how_to_find_good_references?×(1.2.5)_how_to_choose_better_paper_book?×(1.2.5.1)_it_is_selected_as_a_textbook_of_university_lectures×(1.2.5.2)_there_is_good_errata×(1.2.5.3)_it_has_been_revised,_or_it_is_a_long_seller×(1.3)_three_methods_of_information_gathering×(1.3.1)_learn_from_what_you_want_to_know×(1.3.1.1)_lazy_evaluation_study_method×(1.3.1.2)_"you_are_not_gonna_need_it"_yagni_principle×(1.3.1.3)_how_matz_read_source_codes×(1.3.2)_requirement_for_learning_from_what_you_want_to_know×(1.3.2.1)_achievement_conditions_are_clear×(1.3.2.2)_goal_is_achievable×(1.3.2.3)_grasp_the_overall_picture_roughly×(1.3.3)_learn_roughly_first×(column)_do_you_need_the_ability_to_find_information_ten_years_later?×(1.3.3.1)_even_the_material_has_more_than_1,000_pages,_the_table_of_contents_is_only_6_pages×(1.3.3.2)_reading_source_code_roughly×(1.3.3.3)_rough_structure_of_documents×(1.3.3.4)_rough_structure_of_a_paper×(1.3.3.5)_civil_code_map×(column)_excerpt_from_civil_code_map×(1.3.4)_learn_from_one_end×(1.3.4.1)_shakyo×(1.3.4.2)_mathematics×(1.3.4.3)_divide_the_task_by_time×(1.3.4.4)_shakyo_is_an_auxiliary_wheel×(1.3.4.5)_when_you_need_shakyo_again×(1.4)_what_is_abstraction?×(1.4.1)_abstract×(1.4.2)_model×(1.4.3)_module×(1.4.3.1)_restrict_interaction×(1.4.3.2)_hide_non-critical_parts_=_extract_important_parts×(1.4.4)_model_/_view_/_controller×(1.4.5)_pattern_discovery×(1.4.6)_design_pattern×(column)_naming_the_pattern×(1.4.7)_why_is_abstraction_necessary?×(1.4.7.1)_generalization_by_pattern_discovery×(1.5)_how_to_abstract×(1.5.1)_compare_and_learn×(1.5.1.1)_focus_between_'same'_and_'different'×(1.5.1.2)_metaphor×(1.5.1.3)_focus_on_the_difference×(1.5.2)_learning_from_history×(1.5.3)_learn_from_patterns×(1.6)_verification×(1.6.1)_varification_by_making×(1.6.1.1)_explanation_is_a_type_of_making×(1.6.2)_varification_by_exams×(1.6.3)_domains_that_are_difficult_to_verify×(1.7)_summary×(2)_how_to_motivate_yourself×(2.1)_65%_of_not-motivated_people_have_more_than_one_task×(2.1.1)_grasp_the_overall_picture_first_to_choose_one_task×(2.1.2)_getting_things_done:_collect_all_first×(2.1.3)_collect_all_and_then_process_them×(2.1.4)_how_to_choose_one_task×(2.1.4.1)_it_is_similar_to_tidying_up_the_room×(2.1.4.2)_make_a_base×(2.1.4.3)_too_many_tasks×(2.2)_prioritization_of_tasks_is_itself_a_difficult_task×(2.2.1)_a_burden_of_sorting×(2.2.2)_we_can_not_compare_magnitude_unless_it_is_one_dimension×(2.2.3) We can not compare magnitude when there is an uncertain factor?×(2.2.3.1) Exploration-exploitation tradeoff×(2.2.3.2) Optimism in face of uncertainty×(2.2.3.2-2) UCB1 algorithm×(2.2.3.3)_risks_and_values_and_priorities×(2.2.4)_prioritize_important_tasks×(2.2.4.1)_"recently_notified"_is_not_"urgent"×(2.2.4.2)_missions_are_verbalized_bottom-up×(column)_7_habits×(2.2.5)_you_do_not_have_to_determine_the_priority_now×(2.3)_motivation_on_one_task×(2.3.1)_task_is_too_big×(2.3.1.1)_large_task_of_writing_a_book×(2.3.2)_timeboxing×(2.3.2.1)_limit_of_concentration×(2.3.2.1-2)_metaphor_of_hiking×(2.3.2.2)_pomodoro_technique×(2.3.2.3)_train_estimation_skill×(2.3.2.4)_taskchute:_method_to_estimate_time_in_minutes×(column)_pdca_cycle×(2.3.2.5)_measure,_dismiss,_put_together×(2.4)_summary×(3)_how_to_train_your_memory×(3.1)_memory_mechanism×(3.1.1)_hippocampus×(3.1.2)_person_who_removed_hippocampus×(3.1.3)_morris_water_maze×(3.1.4)_memory_is_not_one_type×(3.2)_the_common_part_between_memory_and_muscle×(3.2.1)_synapse_transmitting_a_signal×(3.2.2)_long-term_potentiation_of_synapses×(3.2.3)_gradually_make_the_memory_solid×(3.3)_memory_becomes_strong_by_repeated_use×(column)_in_the_hippocampus_time_is_compressed×(3.4)_the_output_make_memory_strong×(3.4.1)_test_is_a_means_of_memorization×(3.4.2)_learn_more_after_testing×(3.4.3)_not_confident_but_the_score_is_high×(3.4.4)_adaptive_boosting×(3.4.5)_high-speed_cycle_of_test×(3.5)_spaced_repetition_method_that_lasts_knowledge×(3.5.1)_review_after_you_forget_it×(3.5.2)_leitner_system×(3.5.3)_the_easiness_of_the_problem×(3.5.4)_the_20_rules_to_structure_knowledge×(3.5.5)_anki×(3.5.6)_automatic_adjustment_of_difficulty_level×(3.5.7)_make_teaching_materials_yourself×(column)_the_remaining_15_rules_to_structure_knowledge×(3.5.7.1)_making_questions_is_chance_to_make_memory×(3.5.7.2)_use_personal_experiences×(3.5.7.3)_copyright_and_private_use×(3.6)_summary×(4)_how_to_read_efficiently×(4.1)_what_is_"reading?"×(4.1.1)_purpose_of_reading_a_book×(4.1.1.1)_entertainment_is_out_of_scope×(4.1.1.2)_is_the_purpose_to_obtain_information?×(4.1.1.3)_history_of_information_transmission×(4.1.1.4)_assemble_one-dimensional_information_in_the_brain×(4.1.1.5)_only_the_content_of_the_book_is_not_a_box_to_build_understanding×(4.1.1.6)_gradation_between_"finding"_and_"assembling"×(4.1.2)_kind_and_speed_of_reading×(4.2)_how_fast_do_you_read?×(4.2.1)_pyramid_of_reading_speed×(4.2.2)_where_is_the_bottleneck?×(4.2.3)_suffering_of_speed_reading×(4.2.3.1)_know_the_pace_which_you_can_keep×(4.2.4)_to_not_read×(4.2.4.1)_get_information_without_reading×(4.3)_how_to_read_a_page_in_two_seconds_to_find_information×(4.3.1)_whole_mind_system×(4.3.1.1)_❶_preparation×(4.3.1.2)_❷_preview×(4.3.1.3)_❸_photo_reading×(4.3.1.4)_❹_make_questions×(4.3.1.5)_❺_mature×(4.3.1.6)_❻_find_the_answer×(4.3.1.7)_❼_make_a_mind_map×(4.3.1.8)_❽_high-speed_reading×(4.3.1.9)_five_days_training×(4.3.2)_focus_reading×(4.3.2.1)_comparison_between_reading_and_business×(4.3.2.2)_value_of_reading_is_the_product_of_three_power×(4.3.2.3)_measure_and_control_speed×(4.3.3)_attention_to_headlines×(column)_reading_along_time_series×(4.4)_reading_one_page_in_three_minutes_to_assemble_information×(4.4.1)_how_to_read_philosophical_books×(4.4.1.1)_open_book_and_closed_book×(4.4.1.2)_books_that_require_external_reference×(4.4.1.3)_climbing_books_and_hiking_books×(4.4.2)_spend_40_hours_in_one_book×(4.4.2.1)_read_the_shelf×(4.4.2.2)_read_to_make_a_note×(4.4.2.3)_read_to_eliminate_things_what_you_do_not_understand×(4.4.3)_how_to_read_mathematical_books×(4.4.3.1)_definition_of_understanding×(4.4.3.2)_is_it_necessary_to_understand?×(4.5)_design_task_of_reading×(4.5.1)_understanding_is_an_uncertain_task×(4.5.2)_reading_is_a_means,_not_a_purpose×(4.5.2.1)_make_a_map×(4.5.2.2)_make_connections×(4.5.2.3)_get_the_tools_of_thinking×(4.5.3)_make_materials_for_review×(4.5.3.1)_make_leverage_memo×(4.5.3.1-2)_my_failure:_a_service_to_help_review×(4.5.3.2)_incremental_reading×(4.5.3.3)_make_teaching_materials×(4.6)_summary×(5)_how_to_organize_information×(5.1)_is_there_too_much_information_or_too_little?×(5.1.1)_measure_the_amount_of_information_by_writing_all_out×(5.1.1.1)_do_not_care_about_quality×(5.1.1.2)_let's_practice×(5.1.1.3)_put_a_goal_at_100_pieces×(5.1.1.4)_benefits_of_100-piece_goal×(5.1.1.5)_do_not_mind_duplication×(5.2)_how_to_organize_too_much_information×(5.2.1)_spread_so_that_you_can_see_the_whole_at_a_glance×(5.2.2)_record_anything_you_think×(5.2.3)_make_related_things_close×(5.2.3.1)_flow_of_kj_method×(5.2.3.1-1)_exploration_before_starting_the_kj_method×(5.2.3.1-2)_group_organization×(5.2.3.1-3)_illustration_and_documentation×(5.2.3.1-4)_effect_of_changing_format×(5.2.4)_you_need_to_change_your_mental_model_for_group_organization×(5.2.4.0)_concrete_example_of_group_organization×(5.2.4.1)_group_organization_is_not_objective×(5.2.4.2)_group_organization_is_not_a_hierarchical_classification×(5.2.4.3)_problem_of_using_existing_classification_criteria×(5.2.4.4)_problem_of_making_classification_criteria_in_advance×(5.2.4.5)_benefits_of_reducing_burden_by_classification×(5.2.4.6)_family_resemblance×(5.2.5)_what_is_the_relation?×(5.2.5.1)_relationship_is_not_similarity×(5.2.5.1-2)_not_"related_pieces"_but_"pieces_likely_to_be_related"×(5.2.5.2)_focus_on_conflict×(5.2.5.2-2)_conflict_is_not_only_one×(5.2.5.3)_relation_which_connects_story×(5.2.5.4)_group_organization_is_similar_to_method_extraction×(5.2.6)_bundle_and_attach_a_nameplate_=_compress×(5.2.6.1)_merits_and_demerits_of_nameplate_making×(5.2.6.2)_a_group_that_can_make_a_nameplate_is_a_good_group×(5.2.6.3)_making_a_nameplate_when_we_have_a_huge_number_of_pieces×(5.2.6.4)_"organizing_information"_and_"tidying_your_room"_are_similar×(5.2.7)_spread_bundles_again×(5.2.8)_convert_them_into_one-dimensional_sentences×(5.3)_tuning_for_busy_people×(5.3.1)_skip_steps×(5.3.1.1)_skip_nameplate_making×(5.3.1.2)_skip_illustration×(5.3.2)_interruptible_design×(5.3.3)_method_to_store_a4_documents×(5.4)_it_is_important_to_repeat×(5.4.1)_repeating_the_kj_method×(5.4.2)_trigger_to_repeat×(5.4.3)_incremental_improvement×(5.4.4)_organizing_group_of_past_output_again×(5.4.5)_digital_tool_for_the_kj_method×(5.5)_summary×(6)_how_to_come_up_with_ideas×(6.1)__"to_come_up_with_ideas"_is_ambiguous_and_a_big_task×(6.1.1)_three_phases_to_come_up_with_ideas×(6.1.1.2) Seedling Phase×(6.1.2)_predecessor's_idea_creation_methods×(6.1.2.1)_young's_method_to_make_ideas×(6.1.2.2)_jiro_kawakita's_hassouhou×(6.1.2.3)_otto_scharmer's_patterns_of_change×(6.1.2.4)_we_can_not_manage_seedling×(6.2.1)_explore_inside_you×(6.2.2)_how_to_promote_verbalization×(6.2.2.3)_creation_is_subjective×(6.2.3)_physical_sensation×(6.2.3.1)_draw_a_picture×(6.2.4)_parable,_metaphor,_analogy×(6.2.4.1)_nm_method_and_analogy×(6.2.4.2)_clean_language_and_symbolic_modeling×(6.2.5)_things_that_have_not_become_words_yet×(6.2.5.1)_tacit_knowledge:_a_sense_approaching_solution×(6.2.5.2)_discomfort_is_an_important_sign×(6.2.5.3)_thinking_at_the_edge:_where_words_are_still_missing×(6.2.5.4)_matching_with_a_dictionary×(6.2.5.5)_public_words_and_private_words×(6.2.5.6)_use_disconfort_to_arrange_cards×(6.2.6)_summary_of_verbalization×(6.3)_polish_up×(6.3.1)_minimum_viable_product×(6.3.2)_climb_the_u-curve×(6.3.3)_the_viewpoint_of_others_is_important×(6.3.4)_you_can_learn_from_anyone×(6.3.5)_a_customer_want_a_time_machine×(6.3.6)_plow_it_again×(6.4)_summary×(7)_how_to_decide_what_to_learn×(7.1)_what_is_the_right_thing_to_learn?×(7.1.1)_"right"_in_mathematics×(7.1.2)_the_difference_of_"right"_between_science_and_mathematics×(7.1.3)_"right"_in_decision_making×(7.1.3.1)_repetitive_scientific_experiments_and_one-time_decision_making×(7.1.3.2)_"right"_in_decision_making_determined_afterward×(7.1.3.3)_we_can_only_connect_dots_looking_backwards×(7.2)_management_strategy_of_oneself×(7.2.1) Exploring strategies to find targets you want to learn×(7.2.2) Expanded reproduction strategy using knowledge×(7.2.3)_differentiation_strategy_aiming_for_excellence×(7.2.3.1)_getting_knowledge_from_others_is_low_cost×(7.2.3.2)_knowledge_gained_from_others_has_low_value×(7.2.3.3)_to_strive_for_excellence×(7.2.4)_differentiation_strategy_by_crossover×(7.2.4.1)_knowledge_of_two_peaks×(7.2.4.2)_serial_mastery×(7.2.4.3)_an_example_strategy_for_new_employees×(7.2.5)_a_trader_strategy_to_trade_knowledge_crossing_the_boundary_of_the_organization×(7.3)_creating_knowledge×(column)_consistency_of_knowledge×(column)_efficiency_improvement_by_framework×(column)_emergency_decomposition_theory×(column)_example_of_"to_write_all_out"_method×(column)_nameplate_and_color_of_pieces×(column)_size_of_pieces×(column)_you_may_find_a_relationship_later×(column)_knowledge_distribution_chart×(column)_the_impact_of_the_number_of_choices_on_the_quality_of_decision_making×(column)_two_kinds_of_tacit_knowledge×(column)_we_can_not_communicate_bi-directionally_with_books→
- →(7.1.3.2)_"right"_in_decision_making_determined_afterward×explore×exploration-exploitation_tradeoff→
- →(2.2.3.1) Exploration-exploitation tradeoff×expanded_reproduction_of_knowledge×expanded_reproduction×(2.2.4)_prioritize_important_tasks→
- →seedling_phase×by_volume×by_time×(2.3.2)_timeboxing×(2.2.3.1) Exploration-exploitation tradeoff×(3.5)_spaced_repetition_method_that_lasts_knowledge×(4.3.1.5)_❺_mature→
- →seeing×sensing×downloading×exploration-exploitation_tradeoff×(2.2.3.1) Exploration-exploitation tradeoff×jiro_kawakita×todo→
- →variance×pessimistic_misunderstanding×(2.2.3.1) Exploration-exploitation tradeoff×(2.2.3.2) Optimism in face of uncertainty×(2.2.3.3)_risks_and_values_and_priorities×result_of_the_agent_simulation→
- →optimism_in_the_face_of_uncertainty×(2.2.3) We can not compare magnitude when there is an uncertain factor?×pessimistic_misunderstanding×optimistic_misunderstanding×(2.2.3.2-2) UCB1 algorithm×regret_analysis×multi-armed_bandit_problem→
"Engineer's way of creating knowledge" the English version of my book is now available on [Engineer's way of creating knowledge]
(C)NISHIO Hirokazu / Converted from [Scrapbox] at [Edit]