{"id":3004,"date":"2022-06-03T12:38:44","date_gmt":"2022-06-03T03:38:44","guid":{"rendered":"https:\/\/www.ozlab.org\/?page_id=3004"},"modified":"2022-07-29T13:59:57","modified_gmt":"2022-07-29T04:59:57","slug":"stat","status":"publish","type":"page","link":"https:\/\/www.ozlab.org\/en\/stat\/","title":{"rendered":"Stochastic Data Processing"},"content":{"rendered":"<h1 class=\"wp-block-heading\">Stochastic Data Processing<\/h1>\n\n\n\n<hr class=\"wp-block-separator has-css-opacity\"\/>\n\n\n\n<br>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-vertically-aligned-top is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:20%\">\n<div id=\"link\">\n           \n<ul>\n<a href=\"..\/ai\">Artificial Intelligence for Social Issues<\/a>\n<a href=\"..\/iui\">Intelligent User Interface<\/a>\n<a href=\"..\/cscw-2\">Online Class Support<\/a>\n<a href=\"..\/data\">Natural Language Processing<\/a>\n<a href=\"..\/info-2\">Information Recommendation<\/a>\n<a href=\"..\/stat\">Stochastic Data Processing<\/a>\n            <\/ul>\n            \n        <\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:75%\">\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:20%\">\n<h4 class=\"has-text-align-center wp-block-heading\">Members<\/h4>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.66%\">\n<p>R.Kimura, Y.Kusakabe<\/p>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:20%\">\n<h4 class=\"has-text-align-center wp-block-heading\">Purpose<\/h4>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.66%\">\n<p>Proposal of conservative estimation for statistics, development of application applying conservative estimation<\/p>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:20%\">\n<h4 class=\"has-text-align-center wp-block-heading\">Keyword<\/h4>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-vertically-aligned-center is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.66%\">\n<p>Conservative estimation, Observation frequency, Conditional probability, Likelihood ratio<\/p>\n<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:20%\">\n<h4 class=\"has-text-align-center wp-block-heading\">Summary<\/h4>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:66.7%\">\n<p>Statistical estimation using the frequency of observation of events can be easily realized on a computer, and is often performed even recently when large-scale data is handled (For example, the probability of occurrence of a word is presumed from the number of occurrences of the word in the text). Most of these estimation use an unbiased estimator. However, when estimating from a low frequency, the estimated value becomes unstable and the estimated value may be overestimated in using an unbiased estimator. Therefore, it is often devised to estimate the statistic only from the frequency above the threshold, but this method cannot handle low-frequency events below the threshold.<br>We devised an approach called a \"Conservative Estimation\", in which the estimated values \u200b\u200bare intentionally biased to a lower level according to the frequency. So far, we have proposed a conservative estimation for two statistics, conditional probability and likelihood ratio. Furthermore, we applied conservative estimation  to various practical tasks such as association rule mining, named-entity recognition, and multi-armed bandit problem, and confirmed their effectiveness. The conservative estimation makes it possible to treat high-frequency events with priority and statistically treat low-frequency but important events without ignoring them. Future prospects include the realization of conservative estimation for other statistics and the development of applications centered on conservative estimation.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>\u78ba\u7387\u7684\u30c7\u30fc\u30bf\u51e6\u7406 \u793e\u4f1a\u554f\u984c\u89e3\u6c7a\u306e\u4eba\u5de5\u77e5\u80fd \u77e5\u7684\u30e6\u30fc\u30b6\u30a4\u30f3\u30bf\u30d5\u30a7\u30fc\u30b9 \u30aa\u30f3\u30e9\u30a4\u30f3\u6388\u696d\u652f\u63f4 \u81ea\u7136\u8a00\u8a9e\u51e6\u7406\u5fdc\u7528 \u60c5\u5831\u63a8\u85a6\u30fb\u60c5\u5831\u691c\u7d22 \u78ba\u7387\u7684\u30c7\u30fc\u30bf\u51e6\u7406 \u30e1\u30f3\u30d0\u30fc \u6728\u6751\uff0c\u65e5\u4e0b\u90e8 \u76ee\u7684 \u7d71\u8a08\u91cf\u306b\u5bfe\u3059\u308b\u4fdd\u5b88\u7684\u306a\u63a8\u5b9a\u6cd5\u306e\u63d0\u6848\uff0c\u4fdd\u5b88\u7684\u306a\u63a8\u5b9a\u6cd5\u3092\u5fdc\u7528\u3057\u305f\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u306e\u958b\u767a \u30ad\u30fc\u30ef\u30fc\u30c9 \u4fdd\u5b88\u7684\u306a\u63a8\u5b9a\u6cd5\uff0c\u89b3\u6e2c\u983b\u5ea6\uff0c\u6761\u4ef6\u4ed8\u304d\u78ba\u7387\uff0c\u5c24\u5ea6\u6bd4 \u6982\u8981 \u4e8b\u8c61\u306e\u89b3\u6e2c\u983b\u5ea6\u3092\u7528\u3044\u305f\u7d71\u8a08\u91cf\u63a8\u5b9a\u306f\u30b3\u30f3\u30d4\u30e5\u30fc\u30bf\u4e0a\u3067\u7c21\u5358\u306b\u5b9f\u73fe\u3067\u304d\uff0c\u5927\u898f\u6a21\u30c7\u30fc\u30bf\u3092\u6271\u3046\u3088\u3046\u306b\u306a\u3063\u305f\u6700\u8fd1\u3067\u3082\u3088\u304f\u884c\u308f\u308c\u308b\uff08\u4f8b\uff1a\u30c6\u30ad\u30b9\u30c8\u4e2d\u306e\u5358\u8a9e\u306e\u51fa\u73fe\u56de\u6570\u304b\u3089\u305d\u306e\u5358\u8a9e\u306e\u51fa\u73fe\u78ba\u7387\u3092\u63a8\u5b9a\u3059\u308b\uff09\uff0e\u3053\u306e\u3088\u3046\u306a\u63a8\u5b9a\u6cd5\u306e\u307b\u3068\u3093\u3069\u306f\uff0c\u504f\u308a\u306e\u306a\u3044\u63a8\u5b9a\u91cf\u3067\u3042\u308b\u4e0d\u504f\u63a8\u5b9a\u91cf\u3092\u4f7f\u7528\u3057\u3066\u3044\u308b\uff0e\u3057\u304b\u3057\u4e0d\u504f\u63a8\u5b9a\u91cf\u306b\u3088\u308b\u63a8\u5b9a\u6cd5\u306f\uff0c\u4f4e\u983b\u5ea6\u304b\u3089\u63a8\u5b9a\u3092\u884c\u3046\u5834\u5408\uff0c\u63a8\u5b9a\u5024\u304c\u4e0d\u5b89\u5b9a\u306b\u306a\u308a\uff0c\u63a8\u5b9a\u5024\u3092\u904e\u5927\u306b\u898b\u7a4d\u3082\u3063\u3066\u3057\u307e\u3046\u3053\u3068\u304c\u3042\u308b\uff0e\u305d\u306e\u305f\u3081\uff0c\u3057\u304d\u3044\u5024\u4ee5\u4e0a\u306e\u983b\u5ea6\u306e\u307f\u304b\u3089\u7d71\u8a08\u91cf\u3092\u63a8\u5b9a\u3059\u308b\u5de5\u592b\u304c\u3088\u304f\u884c\u308f\u308c\u308b\u304c\uff0c\u3053\u306e\u65b9\u6cd5\u3067\u306f\u3057\u304d\u3044\u5024\u672a\u6e80\u306e\u4f4e\u983b\u5ea6\u4e8b\u8c61\u3092\u6271\u3048\u306a\u3044\uff0e\u3000\u6211\u3005\u306f\u983b\u5ea6\u306e\u4f4e\u3055\u306b\u5fdc\u3058\u3066\u63a8\u5b9a\u5024\u3092\u3042\u3048\u3066\u4f4e\u3081\u306b\u504f\u3089\u305b\u308b\u201c\u4fdd\u5b88\u7684\u306a\u63a8\u5b9a\u6cd5\u201d\u3068\u3044\u3046\u30a2\u30d7\u30ed\u30fc\u30c1\u3092\u8003\u6848\u3057\u305f\uff0e\u305d\u3057\u3066\u3053\u308c\u307e\u3067\u306b\u6761\u4ef6\u4ed8\u304d\u78ba\u7387\uff0c\u5c24\u5ea6\u6bd4\u3068\u3044\u3046\u4e8c\u3064\u306e\u7d71\u8a08\u91cf\u306b\u5bfe\u3057\u3066\uff0c\u4fdd\u5b88\u7684\u306a\u63a8\u5b9a\u6cd5\u3092\u63d0\u6848\u3057\u3066\u3044\u308b\uff0e\u3055\u3089\u306b\uff0c\u76f8\u95a2\u30eb\u30fc\u30eb\u30de\u30a4\u30cb\u30f3\u30b0\uff0c\u56fa\u6709\u8868\u73fe\u62bd\u51fa\uff0c\u30d0\u30f3\u30c7\u30a3\u30c3\u30c8\u554f\u984c\u3068\u3044\u3063\u305f\u7a2e\u3005\u306e\u5b9f\u7528\u30bf\u30b9\u30af\u3078\u3068\u4fdd\u5b88\u7684\u306a\u63a8\u5b9a\u6cd5\u3092\u5fdc\u7528\u3057\uff0c\u305d\u306e\u6709\u52b9\u6027\u3092\u78ba\u8a8d\u3057\u305f\uff0e\u4fdd\u5b88\u7684\u306a\u63a8\u5b9a\u6cd5\u306b\u3088\u3063\u3066\uff0c\u9ad8\u983b\u5ea6\u306e\u4e8b\u8c61\u3092\u512a\u5148 [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-3004","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.ozlab.org\/en\/wp-json\/wp\/v2\/pages\/3004","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.ozlab.org\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.ozlab.org\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.ozlab.org\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.ozlab.org\/en\/wp-json\/wp\/v2\/comments?post=3004"}],"version-history":[{"count":12,"href":"https:\/\/www.ozlab.org\/en\/wp-json\/wp\/v2\/pages\/3004\/revisions"}],"predecessor-version":[{"id":3272,"href":"https:\/\/www.ozlab.org\/en\/wp-json\/wp\/v2\/pages\/3004\/revisions\/3272"}],"wp:attachment":[{"href":"https:\/\/www.ozlab.org\/en\/wp-json\/wp\/v2\/media?parent=3004"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}